Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.305827
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.237889
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.876572
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.820742
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.790998
Train Epoch: 1 - Loss: 1.1009, Accuracy: 78.56%

Test set: Average loss: 0.7660, Accuracy: 9180/10000 (91.80%)

New best model saved with accuracy: 91.80%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.878319
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.797924
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.748512
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.667522
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.718370
Train Epoch: 2 - Loss: 0.7285, Accuracy: 92.95%

Test set: Average loss: 0.6745, Accuracy: 9484/10000 (94.84%)

New best model saved with accuracy: 94.84%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.696441
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.700966
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.673808
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.721056
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.623265
Train Epoch: 3 - Loss: 0.6557, Accuracy: 95.50%

Test set: Average loss: 0.6255, Accuracy: 9647/10000 (96.47%)

New best model saved with accuracy: 96.47%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.612419
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.671697
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.598368
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.608916
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.667202
Train Epoch: 4 - Loss: 0.6186, Accuracy: 96.74%

Test set: Average loss: 0.6019, Accuracy: 9713/10000 (97.13%)

New best model saved with accuracy: 97.13%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.615244
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.624229
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.618509
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.570436
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.605547
Train Epoch: 5 - Loss: 0.5991, Accuracy: 97.33%

Test set: Average loss: 0.5882, Accuracy: 9747/10000 (97.47%)

New best model saved with accuracy: 97.47%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.576787
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.585443
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.575258
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.565036
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.599630
Train Epoch: 6 - Loss: 0.5866, Accuracy: 97.71%

Test set: Average loss: 0.5798, Accuracy: 9783/10000 (97.83%)

New best model saved with accuracy: 97.83%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.581629
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.566044
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.602876
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.593561
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.587602
Train Epoch: 7 - Loss: 0.5779, Accuracy: 97.93%

Test set: Average loss: 0.5740, Accuracy: 9793/10000 (97.93%)

New best model saved with accuracy: 97.93%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.559715
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.560308
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.562651
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.553783
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.564785
Train Epoch: 8 - Loss: 0.5715, Accuracy: 98.15%

Test set: Average loss: 0.5680, Accuracy: 9821/10000 (98.21%)

New best model saved with accuracy: 98.21%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.560829
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.581428
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.591582
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.549217
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.579687
Train Epoch: 9 - Loss: 0.5665, Accuracy: 98.27%

Test set: Average loss: 0.5634, Accuracy: 9827/10000 (98.27%)

New best model saved with accuracy: 98.27%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.607245
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.571210
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.552005
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.608553
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.568449
Train Epoch: 10 - Loss: 0.5625, Accuracy: 98.36%

Test set: Average loss: 0.5634, Accuracy: 9836/10000 (98.36%)

New best model saved with accuracy: 98.36%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.587911
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.558711
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.555112
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.541819
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.539312
Train Epoch: 11 - Loss: 0.5591, Accuracy: 98.51%

Test set: Average loss: 0.5595, Accuracy: 9839/10000 (98.39%)

New best model saved with accuracy: 98.39%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.560143
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.554792
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.533589
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.545892
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.534447
Train Epoch: 12 - Loss: 0.5564, Accuracy: 98.60%

Test set: Average loss: 0.5560, Accuracy: 9842/10000 (98.42%)

New best model saved with accuracy: 98.42%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.534120
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.555119
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.536881
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.562585
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.577629
Train Epoch: 13 - Loss: 0.5544, Accuracy: 98.63%

Test set: Average loss: 0.5557, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.554028
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.556058
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.524194
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.544275
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.545184
Train Epoch: 14 - Loss: 0.5524, Accuracy: 98.72%

Test set: Average loss: 0.5538, Accuracy: 9848/10000 (98.48%)

New best model saved with accuracy: 98.48%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.560410
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.551571
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.544026
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.545969
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.529114
Train Epoch: 15 - Loss: 0.5509, Accuracy: 98.74%

Test set: Average loss: 0.5526, Accuracy: 9841/10000 (98.41%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.533738
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.534240
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.543128
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.538919
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.534407
Train Epoch: 16 - Loss: 0.5497, Accuracy: 98.78%

Test set: Average loss: 0.5519, Accuracy: 9853/10000 (98.53%)

New best model saved with accuracy: 98.53%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.549192
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.531656
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.546564
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.540623
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.535930
Train Epoch: 17 - Loss: 0.5486, Accuracy: 98.81%

Test set: Average loss: 0.5512, Accuracy: 9853/10000 (98.53%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.541739
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.540948
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.532861
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.548517
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.553732
Train Epoch: 18 - Loss: 0.5477, Accuracy: 98.86%

Test set: Average loss: 0.5501, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.541018
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.546155
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.564503
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.576080
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.557128
Train Epoch: 19 - Loss: 0.5471, Accuracy: 98.88%

Test set: Average loss: 0.5500, Accuracy: 9856/10000 (98.56%)

New best model saved with accuracy: 98.56%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.540334
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.555598
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.521801
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.536363
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.531730
Train Epoch: 20 - Loss: 0.5465, Accuracy: 98.91%

Test set: Average loss: 0.5496, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.550895
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.545813
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.568957
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.554710
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.538232
Train Epoch: 21 - Loss: 0.5461, Accuracy: 98.89%

Test set: Average loss: 0.5496, Accuracy: 9854/10000 (98.54%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.562270
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.550306
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.553131
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.532761
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.555297
Train Epoch: 22 - Loss: 0.5458, Accuracy: 98.90%

Test set: Average loss: 0.5492, Accuracy: 9858/10000 (98.58%)

New best model saved with accuracy: 98.58%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.560012
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.526178
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.559437
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.570750
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.548388
Train Epoch: 23 - Loss: 0.5456, Accuracy: 98.93%

Test set: Average loss: 0.5491, Accuracy: 9854/10000 (98.54%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.542467
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.540170
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.536169
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.535430
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.556146
Train Epoch: 24 - Loss: 0.5455, Accuracy: 98.93%

Test set: Average loss: 0.5491, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.539741
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.541202
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.526942
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.543203
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.553437
Train Epoch: 25 - Loss: 0.5454, Accuracy: 98.93%

Test set: Average loss: 0.5491, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.538153
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.534619
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.537897
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.585392
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.534252
Train Epoch: 26 - Loss: 0.5481, Accuracy: 98.85%

Test set: Average loss: 0.5550, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.574813
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.535406
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.537433
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.532604
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.525410
Train Epoch: 27 - Loss: 0.5461, Accuracy: 98.88%

Test set: Average loss: 0.5490, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.533396
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.586428
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.582968
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.561623
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.525693
Train Epoch: 28 - Loss: 0.5438, Accuracy: 98.95%

Test set: Average loss: 0.5484, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.544616
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.524512
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.572085
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.540867
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.532457
Train Epoch: 29 - Loss: 0.5419, Accuracy: 99.00%

Test set: Average loss: 0.5439, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.530728
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.548591
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.551898
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.520943
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.539989
Train Epoch: 30 - Loss: 0.5403, Accuracy: 99.06%

Test set: Average loss: 0.5441, Accuracy: 9869/10000 (98.69%)

New best model saved with accuracy: 98.69%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.538901
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.538930
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.542446
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.536529
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.538571
Train Epoch: 31 - Loss: 0.5389, Accuracy: 99.06%

Test set: Average loss: 0.5427, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.532939
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.526175
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.536700
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.544382
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.524846
Train Epoch: 32 - Loss: 0.5375, Accuracy: 99.14%

Test set: Average loss: 0.5430, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.545014
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.557474
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.525395
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.536399
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.532480
Train Epoch: 33 - Loss: 0.5363, Accuracy: 99.16%

Test set: Average loss: 0.5409, Accuracy: 9883/10000 (98.83%)

New best model saved with accuracy: 98.83%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.522405
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.543822
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.526463
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.526373
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.521678
Train Epoch: 34 - Loss: 0.5352, Accuracy: 99.19%

Test set: Average loss: 0.5406, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.515121
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.548449
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.525437
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.537014
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.526733
Train Epoch: 35 - Loss: 0.5342, Accuracy: 99.25%

Test set: Average loss: 0.5401, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.537274
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.521756
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.515576
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.543737
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.526122
Train Epoch: 36 - Loss: 0.5333, Accuracy: 99.27%

Test set: Average loss: 0.5399, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.517679
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.519112
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.539608
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.545517
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.515799
Train Epoch: 37 - Loss: 0.5324, Accuracy: 99.28%

Test set: Average loss: 0.5381, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.521895
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.534731
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.524188
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.534673
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.527830
Train Epoch: 38 - Loss: 0.5318, Accuracy: 99.30%

Test set: Average loss: 0.5383, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.554286
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.538294
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.518129
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.514556
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.516208
Train Epoch: 39 - Loss: 0.5311, Accuracy: 99.30%

Test set: Average loss: 0.5373, Accuracy: 9886/10000 (98.86%)

New best model saved with accuracy: 98.86%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.522359
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.523966
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.554182
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.548636
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.520247
Train Epoch: 40 - Loss: 0.5305, Accuracy: 99.34%

Test set: Average loss: 0.5370, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.537469
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.517307
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.533432
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.519212
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.529333
Train Epoch: 41 - Loss: 0.5301, Accuracy: 99.34%

Test set: Average loss: 0.5365, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.523341
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.536981
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.540622
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.527907
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.517046
Train Epoch: 42 - Loss: 0.5297, Accuracy: 99.36%

Test set: Average loss: 0.5365, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.520210
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.536567
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.516725
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.525575
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.555812
Train Epoch: 43 - Loss: 0.5293, Accuracy: 99.38%

Test set: Average loss: 0.5361, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.540664
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.530824
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.527632
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.514037
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.519935
Train Epoch: 44 - Loss: 0.5290, Accuracy: 99.39%

Test set: Average loss: 0.5362, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.519076
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.516847
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.535470
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.522924
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.532612
Train Epoch: 45 - Loss: 0.5288, Accuracy: 99.39%

Test set: Average loss: 0.5360, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.531403
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.524459
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.517902
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.531796
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.524835
Train Epoch: 46 - Loss: 0.5286, Accuracy: 99.41%

Test set: Average loss: 0.5360, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.516094
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.532842
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.536827
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.534936
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.518001
Train Epoch: 47 - Loss: 0.5284, Accuracy: 99.41%

Test set: Average loss: 0.5357, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.548021
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.526312
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.521719
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.511425
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.517599
Train Epoch: 48 - Loss: 0.5283, Accuracy: 99.41%

Test set: Average loss: 0.5357, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.527441
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.521489
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.525149
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.524042
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.522564
Train Epoch: 49 - Loss: 0.5282, Accuracy: 99.41%

Test set: Average loss: 0.5357, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.513257
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.531282
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.538039
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.528696
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.534908
Train Epoch: 50 - Loss: 0.5282, Accuracy: 99.41%

Test set: Average loss: 0.5357, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.511455
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.530807
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.518036
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.533729
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.528517
Train Epoch: 51 - Loss: 0.5303, Accuracy: 99.33%

Test set: Average loss: 0.5372, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.524132
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.546645
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.519677
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.516071
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.526509
Train Epoch: 52 - Loss: 0.5296, Accuracy: 99.34%

Test set: Average loss: 0.5358, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.542122
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.525906
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.511181
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.525884
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.524218
Train Epoch: 53 - Loss: 0.5288, Accuracy: 99.39%

Test set: Average loss: 0.5355, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.533918
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.541192
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.511643
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.519820
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.517402
Train Epoch: 54 - Loss: 0.5280, Accuracy: 99.38%

Test set: Average loss: 0.5343, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.521013
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.517767
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.524291
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.518020
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.518087
Train Epoch: 55 - Loss: 0.5272, Accuracy: 99.43%

Test set: Average loss: 0.5344, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.521970
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.511502
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514399
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.522944
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.525239
Train Epoch: 56 - Loss: 0.5265, Accuracy: 99.44%

Test set: Average loss: 0.5342, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.514819
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.520435
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.527482
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.529567
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.517055
Train Epoch: 57 - Loss: 0.5259, Accuracy: 99.45%

Test set: Average loss: 0.5338, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.528885
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.510595
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.527484
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.527110
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.525999
Train Epoch: 58 - Loss: 0.5253, Accuracy: 99.45%

Test set: Average loss: 0.5330, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.516773
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.530745
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.521827
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.551326
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.521224
Train Epoch: 59 - Loss: 0.5247, Accuracy: 99.52%

Test set: Average loss: 0.5333, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.518126
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.530884
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.536442
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.530023
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.519890
Train Epoch: 60 - Loss: 0.5242, Accuracy: 99.51%

Test set: Average loss: 0.5324, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.514743
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.514725
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.520740
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.512215
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.518087
Train Epoch: 61 - Loss: 0.5236, Accuracy: 99.52%

Test set: Average loss: 0.5327, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.526130
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.529398
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.525328
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.521649
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.549384
Train Epoch: 62 - Loss: 0.5232, Accuracy: 99.55%

Test set: Average loss: 0.5321, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.531967
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.515370
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.527106
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.515052
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.523669
Train Epoch: 63 - Loss: 0.5227, Accuracy: 99.57%

Test set: Average loss: 0.5316, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.524388
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.521168
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.518364
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.508276
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.521556
Train Epoch: 64 - Loss: 0.5224, Accuracy: 99.56%

Test set: Average loss: 0.5319, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.522934
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.519459
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.533078
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.515562
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.525226
Train Epoch: 65 - Loss: 0.5221, Accuracy: 99.59%

Test set: Average loss: 0.5316, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.523702
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.528029
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.515861
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.522945
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.513892
Train Epoch: 66 - Loss: 0.5218, Accuracy: 99.58%

Test set: Average loss: 0.5313, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.512857
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.528625
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.511821
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.519138
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.514573
Train Epoch: 67 - Loss: 0.5215, Accuracy: 99.61%

Test set: Average loss: 0.5310, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.533084
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.515326
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.541695
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.533127
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.525611
Train Epoch: 68 - Loss: 0.5213, Accuracy: 99.60%

Test set: Average loss: 0.5312, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.523601
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.511106
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.514745
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.520352
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.528619
Train Epoch: 69 - Loss: 0.5211, Accuracy: 99.62%

Test set: Average loss: 0.5309, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.521530
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.512533
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.524072
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.517120
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.509691
Train Epoch: 70 - Loss: 0.5210, Accuracy: 99.63%

Test set: Average loss: 0.5308, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.541806
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.508555
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.517006
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.515324
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.510801
Train Epoch: 71 - Loss: 0.5209, Accuracy: 99.64%

Test set: Average loss: 0.5308, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.523164
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.530495
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.511299
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.514339
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.545357
Train Epoch: 72 - Loss: 0.5207, Accuracy: 99.64%

Test set: Average loss: 0.5307, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.509994
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.518232
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.515846
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.509151
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.524344
Train Epoch: 73 - Loss: 0.5207, Accuracy: 99.64%

Test set: Average loss: 0.5307, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.528913
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.509498
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.516168
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.508450
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513895
Train Epoch: 74 - Loss: 0.5206, Accuracy: 99.64%

Test set: Average loss: 0.5307, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.525230
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.544140
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.511388
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.514455
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.526187
Train Epoch: 75 - Loss: 0.5206, Accuracy: 99.65%

Test set: Average loss: 0.5307, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.518545
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.534862
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.515797
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.526861
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.527998
Train Epoch: 76 - Loss: 0.5223, Accuracy: 99.57%

Test set: Average loss: 0.5311, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.518751
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.511112
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.519584
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.523572
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.516477
Train Epoch: 77 - Loss: 0.5218, Accuracy: 99.58%

Test set: Average loss: 0.5312, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.520816
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.520431
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.518893
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.515583
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.523276
Train Epoch: 78 - Loss: 0.5213, Accuracy: 99.59%

Test set: Average loss: 0.5323, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.511790
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.522953
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.527537
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.526438
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.521446
Train Epoch: 79 - Loss: 0.5209, Accuracy: 99.60%

Test set: Average loss: 0.5306, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.519143
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.516354
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.509065
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.511278
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.512823
Train Epoch: 80 - Loss: 0.5205, Accuracy: 99.61%

Test set: Average loss: 0.5301, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.520679
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.524031
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.516075
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.536635
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.522598
Train Epoch: 81 - Loss: 0.5200, Accuracy: 99.64%

Test set: Average loss: 0.5304, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.516853
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.514687
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.526404
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.509457
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.512447
Train Epoch: 82 - Loss: 0.5196, Accuracy: 99.63%

Test set: Average loss: 0.5304, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.513217
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.518102
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.509853
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.511947
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.516899
Train Epoch: 83 - Loss: 0.5192, Accuracy: 99.66%

Test set: Average loss: 0.5294, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.512720
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.520510
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.534618
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.520812
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.509626
Train Epoch: 84 - Loss: 0.5189, Accuracy: 99.66%

Test set: Average loss: 0.5302, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.517487
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.528525
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.517934
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.515062
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.512352
Train Epoch: 85 - Loss: 0.5185, Accuracy: 99.68%

Test set: Average loss: 0.5299, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.510551
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.510439
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.510540
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.516363
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.514569
Train Epoch: 86 - Loss: 0.5182, Accuracy: 99.70%

Test set: Average loss: 0.5287, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.509367
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.515087
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.513137
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.523020
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.509740
Train Epoch: 87 - Loss: 0.5179, Accuracy: 99.69%

Test set: Average loss: 0.5287, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.520242
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.512211
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.517972
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.510936
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.507802
Train Epoch: 88 - Loss: 0.5175, Accuracy: 99.72%

Test set: Average loss: 0.5290, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.523299
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.526469
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.522346
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.512966
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.512013
Train Epoch: 89 - Loss: 0.5173, Accuracy: 99.72%

Test set: Average loss: 0.5288, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.512671
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.511016
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.516656
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.513982
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.518688
Train Epoch: 90 - Loss: 0.5171, Accuracy: 99.73%

Test set: Average loss: 0.5284, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.516996
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.516474
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.525458
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.536803
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.516163
Train Epoch: 91 - Loss: 0.5169, Accuracy: 99.73%

Test set: Average loss: 0.5286, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.522730
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.525813
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.513799
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.511161
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.523320
Train Epoch: 92 - Loss: 0.5167, Accuracy: 99.74%

Test set: Average loss: 0.5288, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.519355
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.516011
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.512946
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.514132
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.507705
Train Epoch: 93 - Loss: 0.5165, Accuracy: 99.73%

Test set: Average loss: 0.5284, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.520481
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.513016
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.550418
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.518798
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.507308
Train Epoch: 94 - Loss: 0.5164, Accuracy: 99.74%

Test set: Average loss: 0.5282, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.509042
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.509741
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.517914
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.516089
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509365
Train Epoch: 95 - Loss: 0.5163, Accuracy: 99.75%

Test set: Average loss: 0.5281, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.519310
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.515430
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.506648
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.507817
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.517437
Train Epoch: 96 - Loss: 0.5162, Accuracy: 99.75%

Test set: Average loss: 0.5281, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.509351
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.519475
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.508377
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.514002
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.512931
Train Epoch: 97 - Loss: 0.5161, Accuracy: 99.75%

Test set: Average loss: 0.5281, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.507744
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.510869
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508549
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.510183
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.521511
Train Epoch: 98 - Loss: 0.5161, Accuracy: 99.75%

Test set: Average loss: 0.5281, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.514080
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.536770
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.511310
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.520415
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.509083
Train Epoch: 99 - Loss: 0.5160, Accuracy: 99.76%

Test set: Average loss: 0.5281, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.512747
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.514738
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.515446
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.516158
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.512796
Train Epoch: 100 - Loss: 0.5160, Accuracy: 99.76%

Test set: Average loss: 0.5281, Accuracy: 9902/10000 (99.02%)


Training completed. Best Accuracy: 99.05%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.309223
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.444293
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.937364
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.821234
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.759471
Train Epoch: 1 - Loss: 1.1573, Accuracy: 76.00%

Test set: Average loss: 0.7612, Accuracy: 9208/10000 (92.08%)

New best model saved with accuracy: 92.08%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.814133
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.780906
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.715932
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.710055
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.719918
Train Epoch: 2 - Loss: 0.7176, Accuracy: 93.23%

Test set: Average loss: 0.6631, Accuracy: 9512/10000 (95.12%)

New best model saved with accuracy: 95.12%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.682637
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.649824
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.692375
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.630864
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.621664
Train Epoch: 3 - Loss: 0.6483, Accuracy: 95.57%

Test set: Average loss: 0.6278, Accuracy: 9638/10000 (96.38%)

New best model saved with accuracy: 96.38%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.679627
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.596613
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.570271
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.639975
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.598459
Train Epoch: 4 - Loss: 0.6149, Accuracy: 96.71%

Test set: Average loss: 0.6029, Accuracy: 9715/10000 (97.15%)

New best model saved with accuracy: 97.15%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.602122
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.578161
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.587077
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.613806
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.560583
Train Epoch: 5 - Loss: 0.5967, Accuracy: 97.32%

Test set: Average loss: 0.5882, Accuracy: 9752/10000 (97.52%)

New best model saved with accuracy: 97.52%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.577529
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.616120
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.594997
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.561770
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.597977
Train Epoch: 6 - Loss: 0.5850, Accuracy: 97.66%

Test set: Average loss: 0.5755, Accuracy: 9788/10000 (97.88%)

New best model saved with accuracy: 97.88%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.600240
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.580922
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.594680
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.570299
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.580889
Train Epoch: 7 - Loss: 0.5763, Accuracy: 97.89%

Test set: Average loss: 0.5794, Accuracy: 9765/10000 (97.65%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.579349
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.552479
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.549429
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.536770
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.544220
Train Epoch: 8 - Loss: 0.5699, Accuracy: 98.10%

Test set: Average loss: 0.5671, Accuracy: 9797/10000 (97.97%)

New best model saved with accuracy: 97.97%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.558465
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.553242
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.554819
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.548521
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.565867
Train Epoch: 9 - Loss: 0.5648, Accuracy: 98.29%

Test set: Average loss: 0.5620, Accuracy: 9823/10000 (98.23%)

New best model saved with accuracy: 98.23%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.564755
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.577604
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.561336
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.551666
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.557804
Train Epoch: 10 - Loss: 0.5607, Accuracy: 98.41%

Test set: Average loss: 0.5585, Accuracy: 9837/10000 (98.37%)

New best model saved with accuracy: 98.37%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.552198
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.544660
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.559747
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.577719
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.556648
Train Epoch: 11 - Loss: 0.5575, Accuracy: 98.52%

Test set: Average loss: 0.5571, Accuracy: 9837/10000 (98.37%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.573762
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.531266
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.533987
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.556957
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.531762
Train Epoch: 12 - Loss: 0.5549, Accuracy: 98.59%

Test set: Average loss: 0.5540, Accuracy: 9851/10000 (98.51%)

New best model saved with accuracy: 98.51%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.582315
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.544177
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.543218
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.541033
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.558807
Train Epoch: 13 - Loss: 0.5527, Accuracy: 98.66%

Test set: Average loss: 0.5527, Accuracy: 9841/10000 (98.41%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.555410
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.543286
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.527936
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.539482
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.528859
Train Epoch: 14 - Loss: 0.5507, Accuracy: 98.72%

Test set: Average loss: 0.5513, Accuracy: 9856/10000 (98.56%)

New best model saved with accuracy: 98.56%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.534531
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.543055
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.564039
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.563802
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.561790
Train Epoch: 15 - Loss: 0.5492, Accuracy: 98.78%

Test set: Average loss: 0.5504, Accuracy: 9853/10000 (98.53%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.537353
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.556478
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.549609
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.595476
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.542283
Train Epoch: 16 - Loss: 0.5479, Accuracy: 98.82%

Test set: Average loss: 0.5495, Accuracy: 9854/10000 (98.54%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.562922
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.574131
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.562508
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.545676
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.562845
Train Epoch: 17 - Loss: 0.5469, Accuracy: 98.85%

Test set: Average loss: 0.5492, Accuracy: 9852/10000 (98.52%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.543663
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.537870
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.549978
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.574914
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.565375
Train Epoch: 18 - Loss: 0.5460, Accuracy: 98.89%

Test set: Average loss: 0.5482, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.565906
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.546633
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.527370
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.551577
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.538677
Train Epoch: 19 - Loss: 0.5454, Accuracy: 98.92%

Test set: Average loss: 0.5479, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.544099
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.563714
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.526379
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.547478
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.537026
Train Epoch: 20 - Loss: 0.5448, Accuracy: 98.93%

Test set: Average loss: 0.5478, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.530825
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.569435
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.583548
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.524262
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.543698
Train Epoch: 21 - Loss: 0.5444, Accuracy: 98.97%

Test set: Average loss: 0.5476, Accuracy: 9863/10000 (98.63%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.526455
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.557642
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.525585
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.551485
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.545193
Train Epoch: 22 - Loss: 0.5441, Accuracy: 98.96%

Test set: Average loss: 0.5472, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.576897
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.537772
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.567984
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.568133
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.563660
Train Epoch: 23 - Loss: 0.5439, Accuracy: 98.97%

Test set: Average loss: 0.5472, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.562299
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.546864
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.551648
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.540412
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.543944
Train Epoch: 24 - Loss: 0.5438, Accuracy: 98.97%

Test set: Average loss: 0.5471, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.524500
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.538343
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.525072
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.553916
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.544029
Train Epoch: 25 - Loss: 0.5437, Accuracy: 98.97%

Test set: Average loss: 0.5471, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.589173
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.550517
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.536267
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.540110
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.534619
Train Epoch: 26 - Loss: 0.5461, Accuracy: 98.87%

Test set: Average loss: 0.5471, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.528093
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.538298
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.551282
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.540347
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.541895
Train Epoch: 27 - Loss: 0.5440, Accuracy: 98.93%

Test set: Average loss: 0.5512, Accuracy: 9852/10000 (98.52%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.535083
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.592230
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.527651
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.537211
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.534275
Train Epoch: 28 - Loss: 0.5418, Accuracy: 98.97%

Test set: Average loss: 0.5447, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.525230
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.525344
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.523537
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.568998
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.520750
Train Epoch: 29 - Loss: 0.5401, Accuracy: 99.07%

Test set: Average loss: 0.5449, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.555466
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.521865
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.544515
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.548469
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.552887
Train Epoch: 30 - Loss: 0.5386, Accuracy: 99.07%

Test set: Average loss: 0.5419, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.520484
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.533413
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.522555
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.517586
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.544914
Train Epoch: 31 - Loss: 0.5372, Accuracy: 99.13%

Test set: Average loss: 0.5412, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.538865
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.525969
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.530571
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.522379
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.532532
Train Epoch: 32 - Loss: 0.5359, Accuracy: 99.16%

Test set: Average loss: 0.5403, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.569856
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.523621
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.523545
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.563477
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.526696
Train Epoch: 33 - Loss: 0.5344, Accuracy: 99.20%

Test set: Average loss: 0.5398, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.518543
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.524133
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.545649
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.525677
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.534393
Train Epoch: 34 - Loss: 0.5334, Accuracy: 99.24%

Test set: Average loss: 0.5397, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.532955
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.533196
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.535932
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.525441
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.532189
Train Epoch: 35 - Loss: 0.5326, Accuracy: 99.26%

Test set: Average loss: 0.5406, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.517617
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.552121
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.529548
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.513570
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.520519
Train Epoch: 36 - Loss: 0.5317, Accuracy: 99.26%

Test set: Average loss: 0.5387, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.527904
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.529777
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.520318
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.528611
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.540750
Train Epoch: 37 - Loss: 0.5310, Accuracy: 99.30%

Test set: Average loss: 0.5374, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.521078
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.531876
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.525074
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.521896
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.534258
Train Epoch: 38 - Loss: 0.5303, Accuracy: 99.32%

Test set: Average loss: 0.5368, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.519297
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.518475
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.543790
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.549336
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.519203
Train Epoch: 39 - Loss: 0.5297, Accuracy: 99.36%

Test set: Average loss: 0.5362, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.540463
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.569859
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.529128
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.526274
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.532181
Train Epoch: 40 - Loss: 0.5292, Accuracy: 99.36%

Test set: Average loss: 0.5367, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.536253
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.548019
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.527356
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.524106
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.519293
Train Epoch: 41 - Loss: 0.5287, Accuracy: 99.37%

Test set: Average loss: 0.5356, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.532068
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.541822
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.515966
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.522463
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.517379
Train Epoch: 42 - Loss: 0.5283, Accuracy: 99.38%

Test set: Average loss: 0.5353, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.549124
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.545809
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.536382
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.519713
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.530071
Train Epoch: 43 - Loss: 0.5280, Accuracy: 99.41%

Test set: Average loss: 0.5353, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.517461
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.530369
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.522555
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.531658
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.552808
Train Epoch: 44 - Loss: 0.5277, Accuracy: 99.40%

Test set: Average loss: 0.5352, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.536965
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.542009
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.519004
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.519113
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.529164
Train Epoch: 45 - Loss: 0.5275, Accuracy: 99.41%

Test set: Average loss: 0.5353, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.533522
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.522951
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.539814
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.528539
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.524444
Train Epoch: 46 - Loss: 0.5273, Accuracy: 99.42%

Test set: Average loss: 0.5349, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.512927
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.533353
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.537426
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.514842
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.534482
Train Epoch: 47 - Loss: 0.5271, Accuracy: 99.42%

Test set: Average loss: 0.5349, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.535619
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.550407
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.525783
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.533913
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.518263
Train Epoch: 48 - Loss: 0.5270, Accuracy: 99.43%

Test set: Average loss: 0.5348, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.523780
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.527971
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.541045
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.528906
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.535280
Train Epoch: 49 - Loss: 0.5270, Accuracy: 99.44%

Test set: Average loss: 0.5348, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.525773
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.512043
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.535534
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.547880
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.528752
Train Epoch: 50 - Loss: 0.5269, Accuracy: 99.44%

Test set: Average loss: 0.5348, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.516570
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.532298
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.536106
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.523356
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.529476
Train Epoch: 51 - Loss: 0.5289, Accuracy: 99.38%

Test set: Average loss: 0.5353, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.521635
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.527304
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.546173
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.538073
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.513997
Train Epoch: 52 - Loss: 0.5282, Accuracy: 99.36%

Test set: Average loss: 0.5355, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.531566
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.522487
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.526060
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.514713
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.525254
Train Epoch: 53 - Loss: 0.5275, Accuracy: 99.40%

Test set: Average loss: 0.5356, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.523795
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.538837
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.521798
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.531341
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.518939
Train Epoch: 54 - Loss: 0.5267, Accuracy: 99.40%

Test set: Average loss: 0.5355, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.523776
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.552220
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.521048
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.521541
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.534891
Train Epoch: 55 - Loss: 0.5260, Accuracy: 99.42%

Test set: Average loss: 0.5339, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.514342
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.513564
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.518993
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.529978
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.517246
Train Epoch: 56 - Loss: 0.5254, Accuracy: 99.45%

Test set: Average loss: 0.5336, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.524984
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.527939
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.516776
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.531635
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.541588
Train Epoch: 57 - Loss: 0.5246, Accuracy: 99.49%

Test set: Average loss: 0.5323, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.513033
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.520740
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.514457
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.511964
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.512648
Train Epoch: 58 - Loss: 0.5241, Accuracy: 99.47%

Test set: Average loss: 0.5320, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.515352
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.522261
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.510697
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.513907
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.516195
Train Epoch: 59 - Loss: 0.5236, Accuracy: 99.50%

Test set: Average loss: 0.5326, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.514739
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.530033
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.536655
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.512687
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.521779
Train Epoch: 60 - Loss: 0.5231, Accuracy: 99.52%

Test set: Average loss: 0.5317, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.515884
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.516287
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.533704
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.508815
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.531009
Train Epoch: 61 - Loss: 0.5226, Accuracy: 99.52%

Test set: Average loss: 0.5320, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.520657
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.516910
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.517825
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.520908
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.516168
Train Epoch: 62 - Loss: 0.5222, Accuracy: 99.55%

Test set: Average loss: 0.5316, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.520204
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.517224
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.513156
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.518390
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.521496
Train Epoch: 63 - Loss: 0.5218, Accuracy: 99.55%

Test set: Average loss: 0.5320, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.519382
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.542293
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.528905
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.526751
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.521573
Train Epoch: 64 - Loss: 0.5215, Accuracy: 99.58%

Test set: Average loss: 0.5311, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.535139
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.534109
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.532984
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.510397
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.512775
Train Epoch: 65 - Loss: 0.5211, Accuracy: 99.59%

Test set: Average loss: 0.5309, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.517273
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.512854
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.525270
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.533992
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.529470
Train Epoch: 66 - Loss: 0.5208, Accuracy: 99.59%

Test set: Average loss: 0.5306, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.513878
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.514231
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.516017
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.520660
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513304
Train Epoch: 67 - Loss: 0.5206, Accuracy: 99.60%

Test set: Average loss: 0.5306, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.517565
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.508213
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.514717
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.533302
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.516543
Train Epoch: 68 - Loss: 0.5204, Accuracy: 99.60%

Test set: Average loss: 0.5304, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.514013
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.519818
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.525646
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.514827
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.523751
Train Epoch: 69 - Loss: 0.5202, Accuracy: 99.61%

Test set: Average loss: 0.5305, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.517190
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.512512
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.541164
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.518536
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.530094
Train Epoch: 70 - Loss: 0.5200, Accuracy: 99.62%

Test set: Average loss: 0.5304, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.518606
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.518711
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.520967
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.509600
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.510804
Train Epoch: 71 - Loss: 0.5199, Accuracy: 99.62%

Test set: Average loss: 0.5303, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.518903
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.520992
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.521182
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512991
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.514115
Train Epoch: 72 - Loss: 0.5198, Accuracy: 99.62%

Test set: Average loss: 0.5304, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.514896
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.525515
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.529794
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.523324
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.519560
Train Epoch: 73 - Loss: 0.5198, Accuracy: 99.62%

Test set: Average loss: 0.5303, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.520409
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.518521
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.513585
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.518231
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.527997
Train Epoch: 74 - Loss: 0.5197, Accuracy: 99.63%

Test set: Average loss: 0.5303, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.533405
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.510408
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.519812
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.519334
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.511920
Train Epoch: 75 - Loss: 0.5197, Accuracy: 99.62%

Test set: Average loss: 0.5303, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.519288
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.512237
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.515927
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.510237
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.514930
Train Epoch: 76 - Loss: 0.5213, Accuracy: 99.56%

Test set: Average loss: 0.5313, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.515054
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.514003
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.518548
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.510418
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.518627
Train Epoch: 77 - Loss: 0.5208, Accuracy: 99.60%

Test set: Average loss: 0.5304, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.506936
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.525022
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.525654
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.517294
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.523142
Train Epoch: 78 - Loss: 0.5205, Accuracy: 99.58%

Test set: Average loss: 0.5300, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.524167
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.533451
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.528594
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.520617
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.524874
Train Epoch: 79 - Loss: 0.5200, Accuracy: 99.60%

Test set: Average loss: 0.5304, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.515081
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.539137
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.513666
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.513664
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.512265
Train Epoch: 80 - Loss: 0.5195, Accuracy: 99.63%

Test set: Average loss: 0.5297, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.515831
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.515669
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.528622
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.518579
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.534444
Train Epoch: 81 - Loss: 0.5191, Accuracy: 99.62%

Test set: Average loss: 0.5300, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.524808
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.509602
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.545725
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.525525
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.514220
Train Epoch: 82 - Loss: 0.5188, Accuracy: 99.64%

Test set: Average loss: 0.5287, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.508212
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.515397
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.512272
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.524347
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.514109
Train Epoch: 83 - Loss: 0.5184, Accuracy: 99.65%

Test set: Average loss: 0.5300, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.518835
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.515972
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.522075
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.538903
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.512236
Train Epoch: 84 - Loss: 0.5180, Accuracy: 99.66%

Test set: Average loss: 0.5296, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.511087
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.521102
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.507513
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.513733
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.548352
Train Epoch: 85 - Loss: 0.5177, Accuracy: 99.69%

Test set: Average loss: 0.5292, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.517409
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.509957
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.515467
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.516042
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.510508
Train Epoch: 86 - Loss: 0.5174, Accuracy: 99.67%

Test set: Average loss: 0.5284, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.517120
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.519573
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.515410
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.508271
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.508761
Train Epoch: 87 - Loss: 0.5170, Accuracy: 99.70%

Test set: Average loss: 0.5284, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.509022
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.527764
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.519549
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.524205
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.515233
Train Epoch: 88 - Loss: 0.5168, Accuracy: 99.70%

Test set: Average loss: 0.5286, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.529133
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.507893
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.521394
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.546397
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.507973
Train Epoch: 89 - Loss: 0.5165, Accuracy: 99.71%

Test set: Average loss: 0.5282, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.508107
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.515394
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.512171
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508687
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.511257
Train Epoch: 90 - Loss: 0.5163, Accuracy: 99.72%

Test set: Average loss: 0.5283, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.531578
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.516838
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.512098
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.525748
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.510302
Train Epoch: 91 - Loss: 0.5161, Accuracy: 99.73%

Test set: Average loss: 0.5282, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.511961
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.517693
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.527846
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.514902
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.512102
Train Epoch: 92 - Loss: 0.5160, Accuracy: 99.73%

Test set: Average loss: 0.5282, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.515587
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.514419
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.518133
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.509248
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.516441
Train Epoch: 93 - Loss: 0.5158, Accuracy: 99.74%

Test set: Average loss: 0.5281, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.514000
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509057
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.522598
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.513788
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.519600
Train Epoch: 94 - Loss: 0.5157, Accuracy: 99.74%

Test set: Average loss: 0.5279, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.519275
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.514430
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.514043
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.520329
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.536960
Train Epoch: 95 - Loss: 0.5156, Accuracy: 99.75%

Test set: Average loss: 0.5279, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.537593
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508145
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507079
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.510039
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.511341
Train Epoch: 96 - Loss: 0.5155, Accuracy: 99.74%

Test set: Average loss: 0.5280, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.511459
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.513154
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.507478
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.512062
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507322
Train Epoch: 97 - Loss: 0.5154, Accuracy: 99.75%

Test set: Average loss: 0.5279, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506534
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.516350
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.527995
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.521087
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.528827
Train Epoch: 98 - Loss: 0.5153, Accuracy: 99.76%

Test set: Average loss: 0.5279, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.512289
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.517221
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.517644
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.515575
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.530051
Train Epoch: 99 - Loss: 0.5153, Accuracy: 99.75%

Test set: Average loss: 0.5279, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.510530
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.511005
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.506949
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.528149
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.515239
Train Epoch: 100 - Loss: 0.5153, Accuracy: 99.76%

Test set: Average loss: 0.5279, Accuracy: 9905/10000 (99.05%)


Training completed. Best Accuracy: 99.06%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.309929
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.749500
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.002999
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.800240
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.760853
Train Epoch: 1 - Loss: 1.1890, Accuracy: 74.90%

Test set: Average loss: 0.7561, Accuracy: 9197/10000 (91.97%)

New best model saved with accuracy: 91.97%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.746829
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.695663
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.743080
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.783201
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.681932
Train Epoch: 2 - Loss: 0.7140, Accuracy: 93.35%

Test set: Average loss: 0.6609, Accuracy: 9516/10000 (95.16%)

New best model saved with accuracy: 95.16%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.634802
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.633674
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.780254
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.635846
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.604013
Train Epoch: 3 - Loss: 0.6465, Accuracy: 95.63%

Test set: Average loss: 0.6187, Accuracy: 9658/10000 (96.58%)

New best model saved with accuracy: 96.58%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.665715
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.626362
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.612149
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.629734
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.645301
Train Epoch: 4 - Loss: 0.6151, Accuracy: 96.73%

Test set: Average loss: 0.6036, Accuracy: 9707/10000 (97.07%)

New best model saved with accuracy: 97.07%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.606132
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.613980
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.582127
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.583662
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.563068
Train Epoch: 5 - Loss: 0.5973, Accuracy: 97.31%

Test set: Average loss: 0.5865, Accuracy: 9776/10000 (97.76%)

New best model saved with accuracy: 97.76%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.576060
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.582186
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.624114
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.618634
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.588973
Train Epoch: 6 - Loss: 0.5855, Accuracy: 97.71%

Test set: Average loss: 0.5777, Accuracy: 9778/10000 (97.78%)

New best model saved with accuracy: 97.78%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.577152
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.573139
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.585534
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.591897
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.587954
Train Epoch: 7 - Loss: 0.5771, Accuracy: 97.98%

Test set: Average loss: 0.5709, Accuracy: 9810/10000 (98.10%)

New best model saved with accuracy: 98.10%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.552912
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.575794
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.572545
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.596482
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.575830
Train Epoch: 8 - Loss: 0.5704, Accuracy: 98.13%

Test set: Average loss: 0.5668, Accuracy: 9817/10000 (98.17%)

New best model saved with accuracy: 98.17%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.572778
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.557759
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.551856
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.542123
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.541374
Train Epoch: 9 - Loss: 0.5656, Accuracy: 98.30%

Test set: Average loss: 0.5619, Accuracy: 9835/10000 (98.35%)

New best model saved with accuracy: 98.35%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.554083
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.580260
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.570646
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.580638
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.555092
Train Epoch: 10 - Loss: 0.5619, Accuracy: 98.40%

Test set: Average loss: 0.5610, Accuracy: 9836/10000 (98.36%)

New best model saved with accuracy: 98.36%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.556561
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.568814
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.560240
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.540781
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.534921
Train Epoch: 11 - Loss: 0.5586, Accuracy: 98.51%

Test set: Average loss: 0.5575, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.560568
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.550587
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.578509
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.534805
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.532314
Train Epoch: 12 - Loss: 0.5561, Accuracy: 98.56%

Test set: Average loss: 0.5562, Accuracy: 9846/10000 (98.46%)

New best model saved with accuracy: 98.46%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.537733
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.563646
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.560032
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.538362
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.542625
Train Epoch: 13 - Loss: 0.5539, Accuracy: 98.61%

Test set: Average loss: 0.5538, Accuracy: 9845/10000 (98.45%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.546080
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.547075
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.559149
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.567348
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.530661
Train Epoch: 14 - Loss: 0.5520, Accuracy: 98.68%

Test set: Average loss: 0.5529, Accuracy: 9848/10000 (98.48%)

New best model saved with accuracy: 98.48%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.562121
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.543069
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.550590
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.550180
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.532688
Train Epoch: 15 - Loss: 0.5505, Accuracy: 98.73%

Test set: Average loss: 0.5517, Accuracy: 9851/10000 (98.51%)

New best model saved with accuracy: 98.51%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.540222
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.529058
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.530905
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.543767
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.573296
Train Epoch: 16 - Loss: 0.5493, Accuracy: 98.77%

Test set: Average loss: 0.5506, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.555703
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.549829
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.555291
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.556972
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.521792
Train Epoch: 17 - Loss: 0.5483, Accuracy: 98.78%

Test set: Average loss: 0.5506, Accuracy: 9851/10000 (98.51%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.537194
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.547483
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.557032
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.530655
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.550603
Train Epoch: 18 - Loss: 0.5475, Accuracy: 98.84%

Test set: Average loss: 0.5498, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.557175
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.543771
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.539767
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.530652
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.538362
Train Epoch: 19 - Loss: 0.5468, Accuracy: 98.85%

Test set: Average loss: 0.5494, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.534952
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.532540
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.552435
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.555816
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.529980
Train Epoch: 20 - Loss: 0.5463, Accuracy: 98.86%

Test set: Average loss: 0.5493, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.533689
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.576742
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.551815
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.566610
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.546482
Train Epoch: 21 - Loss: 0.5459, Accuracy: 98.88%

Test set: Average loss: 0.5490, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.538992
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.560357
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.571553
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.562925
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.555828
Train Epoch: 22 - Loss: 0.5456, Accuracy: 98.88%

Test set: Average loss: 0.5488, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.528808
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.540989
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.559783
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.560180
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.530382
Train Epoch: 23 - Loss: 0.5454, Accuracy: 98.90%

Test set: Average loss: 0.5486, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.544895
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.530923
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.553405
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.543263
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.530314
Train Epoch: 24 - Loss: 0.5452, Accuracy: 98.91%

Test set: Average loss: 0.5486, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.562963
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.533621
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.553916
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.531700
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.524940
Train Epoch: 25 - Loss: 0.5452, Accuracy: 98.89%

Test set: Average loss: 0.5486, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.526237
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.544764
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.552911
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.546086
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.556722
Train Epoch: 26 - Loss: 0.5479, Accuracy: 98.83%

Test set: Average loss: 0.5497, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.563158
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.537257
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.537204
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.553277
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.557436
Train Epoch: 27 - Loss: 0.5459, Accuracy: 98.87%

Test set: Average loss: 0.5472, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.574606
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.553836
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.536397
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.528423
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.532847
Train Epoch: 28 - Loss: 0.5440, Accuracy: 98.91%

Test set: Average loss: 0.5482, Accuracy: 9869/10000 (98.69%)

New best model saved with accuracy: 98.69%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.576002
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.524256
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.547362
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.555022
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.547040
Train Epoch: 29 - Loss: 0.5420, Accuracy: 98.94%

Test set: Average loss: 0.5453, Accuracy: 9869/10000 (98.69%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.520572
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.530496
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.534268
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.526687
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.532261
Train Epoch: 30 - Loss: 0.5405, Accuracy: 98.98%

Test set: Average loss: 0.5443, Accuracy: 9867/10000 (98.67%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.541453
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.539665
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.527738
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.553956
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.537624
Train Epoch: 31 - Loss: 0.5390, Accuracy: 99.05%

Test set: Average loss: 0.5435, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.552303
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.547207
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.532515
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.524547
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.516331
Train Epoch: 32 - Loss: 0.5375, Accuracy: 99.07%

Test set: Average loss: 0.5422, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.515293
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.528330
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.538202
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.525814
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.554644
Train Epoch: 33 - Loss: 0.5363, Accuracy: 99.14%

Test set: Average loss: 0.5424, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.549357
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.544093
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.533933
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.523441
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.526465
Train Epoch: 34 - Loss: 0.5353, Accuracy: 99.13%

Test set: Average loss: 0.5403, Accuracy: 9883/10000 (98.83%)

New best model saved with accuracy: 98.83%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.545167
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.559543
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.534795
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.518619
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.578836
Train Epoch: 35 - Loss: 0.5342, Accuracy: 99.18%

Test set: Average loss: 0.5395, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.530635
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.533702
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.513906
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.547776
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.520827
Train Epoch: 36 - Loss: 0.5335, Accuracy: 99.17%

Test set: Average loss: 0.5389, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.575426
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.541139
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.521851
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.535907
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.525385
Train Epoch: 37 - Loss: 0.5325, Accuracy: 99.23%

Test set: Average loss: 0.5385, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.533289
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.535694
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.516942
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.540196
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.549310
Train Epoch: 38 - Loss: 0.5319, Accuracy: 99.27%

Test set: Average loss: 0.5376, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.543105
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.523229
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.519791
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.547045
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.516095
Train Epoch: 39 - Loss: 0.5312, Accuracy: 99.25%

Test set: Average loss: 0.5384, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.538836
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.528128
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.538465
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.520155
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.521313
Train Epoch: 40 - Loss: 0.5307, Accuracy: 99.31%

Test set: Average loss: 0.5370, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.516382
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.520418
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.535411
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.523636
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.528823
Train Epoch: 41 - Loss: 0.5302, Accuracy: 99.31%

Test set: Average loss: 0.5366, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.518532
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.513978
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.519731
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.528548
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.527583
Train Epoch: 42 - Loss: 0.5298, Accuracy: 99.33%

Test set: Average loss: 0.5367, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.521610
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.564604
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.539276
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.525495
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.551552
Train Epoch: 43 - Loss: 0.5294, Accuracy: 99.33%

Test set: Average loss: 0.5363, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.526696
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.545356
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.517682
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.543218
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.530543
Train Epoch: 44 - Loss: 0.5291, Accuracy: 99.34%

Test set: Average loss: 0.5363, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.522822
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.515070
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.521655
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.516474
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.517403
Train Epoch: 45 - Loss: 0.5289, Accuracy: 99.36%

Test set: Average loss: 0.5360, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.538501
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.526748
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.535144
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.522073
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.543449
Train Epoch: 46 - Loss: 0.5287, Accuracy: 99.38%

Test set: Average loss: 0.5362, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.528451
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.521423
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.522954
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.539518
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.522567
Train Epoch: 47 - Loss: 0.5285, Accuracy: 99.37%

Test set: Average loss: 0.5362, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.527939
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.521130
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.521750
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.529432
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.528566
Train Epoch: 48 - Loss: 0.5284, Accuracy: 99.39%

Test set: Average loss: 0.5360, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.522058
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.512702
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.521124
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.532568
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.519216
Train Epoch: 49 - Loss: 0.5283, Accuracy: 99.39%

Test set: Average loss: 0.5359, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.534344
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.536842
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.532762
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.540588
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.536043
Train Epoch: 50 - Loss: 0.5283, Accuracy: 99.38%

Test set: Average loss: 0.5359, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.531498
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.538186
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.529905
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.554564
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.525462
Train Epoch: 51 - Loss: 0.5304, Accuracy: 99.30%

Test set: Average loss: 0.5359, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.529311
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.526027
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.515878
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.537785
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.515327
Train Epoch: 52 - Loss: 0.5296, Accuracy: 99.33%

Test set: Average loss: 0.5374, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.558537
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.517534
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.519739
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.522803
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.541837
Train Epoch: 53 - Loss: 0.5289, Accuracy: 99.35%

Test set: Average loss: 0.5351, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.528494
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.520058
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.520383
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.518983
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.533742
Train Epoch: 54 - Loss: 0.5281, Accuracy: 99.35%

Test set: Average loss: 0.5361, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.529150
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.516335
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.542925
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.524567
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.523097
Train Epoch: 55 - Loss: 0.5274, Accuracy: 99.39%

Test set: Average loss: 0.5351, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.523007
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.525777
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.544034
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.510884
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.517695
Train Epoch: 56 - Loss: 0.5267, Accuracy: 99.39%

Test set: Average loss: 0.5339, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.520045
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.524867
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.521586
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.511965
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.526877
Train Epoch: 57 - Loss: 0.5260, Accuracy: 99.44%

Test set: Average loss: 0.5329, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.537874
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.517993
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.550584
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.548799
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.518861
Train Epoch: 58 - Loss: 0.5253, Accuracy: 99.43%

Test set: Average loss: 0.5339, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.516523
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.524378
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.513142
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.537765
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.521574
Train Epoch: 59 - Loss: 0.5248, Accuracy: 99.47%

Test set: Average loss: 0.5353, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.520430
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.540059
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.530695
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.525154
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.517128
Train Epoch: 60 - Loss: 0.5242, Accuracy: 99.48%

Test set: Average loss: 0.5324, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.517890
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.520597
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.520108
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.539003
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.520314
Train Epoch: 61 - Loss: 0.5237, Accuracy: 99.51%

Test set: Average loss: 0.5320, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.514456
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.538796
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.512430
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.510164
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.517918
Train Epoch: 62 - Loss: 0.5233, Accuracy: 99.50%

Test set: Average loss: 0.5324, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.524225
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.524410
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.522749
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.524827
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.521559
Train Epoch: 63 - Loss: 0.5228, Accuracy: 99.53%

Test set: Average loss: 0.5319, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.526357
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.509870
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.510565
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.510000
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.512067
Train Epoch: 64 - Loss: 0.5224, Accuracy: 99.52%

Test set: Average loss: 0.5319, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.514705
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.526218
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.513390
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.514953
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.524746
Train Epoch: 65 - Loss: 0.5221, Accuracy: 99.56%

Test set: Average loss: 0.5312, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.522626
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.514469
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.523116
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.517731
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.513073
Train Epoch: 66 - Loss: 0.5218, Accuracy: 99.56%

Test set: Average loss: 0.5309, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.515683
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.511545
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.515428
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.509970
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.520991
Train Epoch: 67 - Loss: 0.5215, Accuracy: 99.55%

Test set: Average loss: 0.5309, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.512376
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.509938
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.532944
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.512450
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.524312
Train Epoch: 68 - Loss: 0.5213, Accuracy: 99.57%

Test set: Average loss: 0.5310, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.513021
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.516680
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.526851
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.512903
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.523121
Train Epoch: 69 - Loss: 0.5211, Accuracy: 99.57%

Test set: Average loss: 0.5313, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.530136
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.514924
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.543385
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.509357
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.531144
Train Epoch: 70 - Loss: 0.5209, Accuracy: 99.58%

Test set: Average loss: 0.5310, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.508902
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.518954
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.517158
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.529181
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.516660
Train Epoch: 71 - Loss: 0.5208, Accuracy: 99.59%

Test set: Average loss: 0.5307, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.525059
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.519620
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509659
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.531936
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.516777
Train Epoch: 72 - Loss: 0.5207, Accuracy: 99.59%

Test set: Average loss: 0.5307, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.516283
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.520686
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.525073
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.530647
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.512971
Train Epoch: 73 - Loss: 0.5206, Accuracy: 99.59%

Test set: Average loss: 0.5306, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.519810
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.509476
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.515713
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.515757
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.531664
Train Epoch: 74 - Loss: 0.5206, Accuracy: 99.59%

Test set: Average loss: 0.5306, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.520478
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.516226
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.513003
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.508359
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.508719
Train Epoch: 75 - Loss: 0.5205, Accuracy: 99.59%

Test set: Average loss: 0.5306, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.518605
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.520026
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.515476
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.527439
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.510338
Train Epoch: 76 - Loss: 0.5224, Accuracy: 99.52%

Test set: Average loss: 0.5308, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.533178
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.521006
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.538571
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.516072
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.542270
Train Epoch: 77 - Loss: 0.5218, Accuracy: 99.54%

Test set: Average loss: 0.5314, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.520262
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.508507
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.519279
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.524997
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.538332
Train Epoch: 78 - Loss: 0.5213, Accuracy: 99.58%

Test set: Average loss: 0.5310, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.515815
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.520644
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.519407
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.512395
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.525361
Train Epoch: 79 - Loss: 0.5210, Accuracy: 99.59%

Test set: Average loss: 0.5325, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.523243
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.515018
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.518333
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.512850
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.532312
Train Epoch: 80 - Loss: 0.5205, Accuracy: 99.59%

Test set: Average loss: 0.5311, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.524595
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.516904
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.521596
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.523957
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.515674
Train Epoch: 81 - Loss: 0.5200, Accuracy: 99.59%

Test set: Average loss: 0.5306, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.519972
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.522614
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.518090
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.531365
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.525202
Train Epoch: 82 - Loss: 0.5197, Accuracy: 99.59%

Test set: Average loss: 0.5292, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.527706
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.516009
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.532677
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.528913
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.513369
Train Epoch: 83 - Loss: 0.5193, Accuracy: 99.61%

Test set: Average loss: 0.5296, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.512035
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.525290
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.526777
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.527662
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.519480
Train Epoch: 84 - Loss: 0.5188, Accuracy: 99.64%

Test set: Average loss: 0.5293, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.526333
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.521198
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.525530
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.508913
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.525177
Train Epoch: 85 - Loss: 0.5185, Accuracy: 99.65%

Test set: Average loss: 0.5298, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.514986
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.524438
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.526676
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.516248
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.511259
Train Epoch: 86 - Loss: 0.5182, Accuracy: 99.64%

Test set: Average loss: 0.5289, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.539894
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.509261
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.524844
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.511578
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.511721
Train Epoch: 87 - Loss: 0.5178, Accuracy: 99.66%

Test set: Average loss: 0.5293, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.519774
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.513840
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.508992
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.512214
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.508916
Train Epoch: 88 - Loss: 0.5175, Accuracy: 99.68%

Test set: Average loss: 0.5289, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.517320
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.528001
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.514276
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.510701
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.515260
Train Epoch: 89 - Loss: 0.5172, Accuracy: 99.67%

Test set: Average loss: 0.5286, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.519631
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.507352
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.513880
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.510510
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.513312
Train Epoch: 90 - Loss: 0.5170, Accuracy: 99.70%

Test set: Average loss: 0.5282, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.514742
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.517603
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.510118
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.512941
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.506851
Train Epoch: 91 - Loss: 0.5168, Accuracy: 99.70%

Test set: Average loss: 0.5284, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.518685
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.515922
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.515714
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.517405
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.516182
Train Epoch: 92 - Loss: 0.5166, Accuracy: 99.71%

Test set: Average loss: 0.5282, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.510137
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.528379
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.508625
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.516144
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.523957
Train Epoch: 93 - Loss: 0.5164, Accuracy: 99.72%

Test set: Average loss: 0.5280, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.520056
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.533225
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.509466
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.523131
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.512187
Train Epoch: 94 - Loss: 0.5163, Accuracy: 99.72%

Test set: Average loss: 0.5280, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.523046
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.521865
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.510544
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.528436
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.514164
Train Epoch: 95 - Loss: 0.5162, Accuracy: 99.72%

Test set: Average loss: 0.5281, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.515945
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.513017
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.514214
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.509950
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.508481
Train Epoch: 96 - Loss: 0.5161, Accuracy: 99.73%

Test set: Average loss: 0.5280, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.513227
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508182
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.518444
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.527394
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.528501
Train Epoch: 97 - Loss: 0.5160, Accuracy: 99.73%

Test set: Average loss: 0.5279, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.515262
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.528482
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.521016
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.517956
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.515395
Train Epoch: 98 - Loss: 0.5159, Accuracy: 99.74%

Test set: Average loss: 0.5279, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.514344
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.508448
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.526055
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.511709
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.515632
Train Epoch: 99 - Loss: 0.5159, Accuracy: 99.74%

Test set: Average loss: 0.5279, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.510421
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.516993
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.520828
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.508454
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.507390
Train Epoch: 100 - Loss: 0.5159, Accuracy: 99.74%

Test set: Average loss: 0.5279, Accuracy: 9904/10000 (99.04%)


Training completed. Best Accuracy: 99.09%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.298902
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.282969
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.910303
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.811921
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.799971
Train Epoch: 1 - Loss: 1.1248, Accuracy: 78.74%

Test set: Average loss: 0.7591, Accuracy: 9206/10000 (92.06%)

New best model saved with accuracy: 92.06%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.793753
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.709585
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.778466
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.768464
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.687344
Train Epoch: 2 - Loss: 0.7196, Accuracy: 93.01%

Test set: Average loss: 0.6699, Accuracy: 9477/10000 (94.77%)

New best model saved with accuracy: 94.77%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.646217
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.670972
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.679069
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.625912
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.698863
Train Epoch: 3 - Loss: 0.6517, Accuracy: 95.31%

Test set: Average loss: 0.6244, Accuracy: 9636/10000 (96.36%)

New best model saved with accuracy: 96.36%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.677638
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.622271
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.613104
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.617136
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.651664
Train Epoch: 4 - Loss: 0.6174, Accuracy: 96.53%

Test set: Average loss: 0.5998, Accuracy: 9709/10000 (97.09%)

New best model saved with accuracy: 97.09%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.623739
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.604733
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.625298
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.606423
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.608088
Train Epoch: 5 - Loss: 0.5979, Accuracy: 97.20%

Test set: Average loss: 0.5885, Accuracy: 9750/10000 (97.50%)

New best model saved with accuracy: 97.50%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.589684
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.550110
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.595041
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.564995
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.582683
Train Epoch: 6 - Loss: 0.5854, Accuracy: 97.66%

Test set: Average loss: 0.5786, Accuracy: 9768/10000 (97.68%)

New best model saved with accuracy: 97.68%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.558780
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.551126
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.581088
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.555061
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.582271
Train Epoch: 7 - Loss: 0.5765, Accuracy: 97.88%

Test set: Average loss: 0.5723, Accuracy: 9797/10000 (97.97%)

New best model saved with accuracy: 97.97%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.559826
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.594379
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.585549
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.567466
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.553248
Train Epoch: 8 - Loss: 0.5700, Accuracy: 98.07%

Test set: Average loss: 0.5688, Accuracy: 9798/10000 (97.98%)

New best model saved with accuracy: 97.98%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.571635
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.575794
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.539578
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.551993
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.653343
Train Epoch: 9 - Loss: 0.5649, Accuracy: 98.27%

Test set: Average loss: 0.5645, Accuracy: 9819/10000 (98.19%)

New best model saved with accuracy: 98.19%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.585429
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.577299
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.579636
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.572458
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.562348
Train Epoch: 10 - Loss: 0.5609, Accuracy: 98.40%

Test set: Average loss: 0.5616, Accuracy: 9820/10000 (98.20%)

New best model saved with accuracy: 98.20%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.580919
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.541849
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.540649
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.548239
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.534489
Train Epoch: 11 - Loss: 0.5575, Accuracy: 98.51%

Test set: Average loss: 0.5581, Accuracy: 9833/10000 (98.33%)

New best model saved with accuracy: 98.33%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.564385
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.531995
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.557846
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.540917
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.541629
Train Epoch: 12 - Loss: 0.5548, Accuracy: 98.58%

Test set: Average loss: 0.5578, Accuracy: 9842/10000 (98.42%)

New best model saved with accuracy: 98.42%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.555153
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.558847
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.527779
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.548863
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.538247
Train Epoch: 13 - Loss: 0.5528, Accuracy: 98.66%

Test set: Average loss: 0.5543, Accuracy: 9850/10000 (98.50%)

New best model saved with accuracy: 98.50%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.555714
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.545935
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.539021
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.565466
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.585575
Train Epoch: 14 - Loss: 0.5509, Accuracy: 98.72%

Test set: Average loss: 0.5536, Accuracy: 9849/10000 (98.49%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.538087
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.536815
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.524832
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.526304
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.580522
Train Epoch: 15 - Loss: 0.5494, Accuracy: 98.76%

Test set: Average loss: 0.5527, Accuracy: 9843/10000 (98.43%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.541596
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.541909
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.561348
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.543596
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.521724
Train Epoch: 16 - Loss: 0.5482, Accuracy: 98.80%

Test set: Average loss: 0.5517, Accuracy: 9847/10000 (98.47%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.545293
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.524689
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.545352
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.543531
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.549842
Train Epoch: 17 - Loss: 0.5471, Accuracy: 98.85%

Test set: Average loss: 0.5506, Accuracy: 9860/10000 (98.60%)

New best model saved with accuracy: 98.60%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.539017
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.559031
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.526938
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.566040
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.547483
Train Epoch: 18 - Loss: 0.5462, Accuracy: 98.87%

Test set: Average loss: 0.5502, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.542100
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.532591
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.560676
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.543010
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.541393
Train Epoch: 19 - Loss: 0.5456, Accuracy: 98.89%

Test set: Average loss: 0.5500, Accuracy: 9854/10000 (98.54%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.520324
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.576236
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.568910
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.546492
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.561223
Train Epoch: 20 - Loss: 0.5451, Accuracy: 98.92%

Test set: Average loss: 0.5494, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.551304
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.549227
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.555472
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.541000
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.565841
Train Epoch: 21 - Loss: 0.5447, Accuracy: 98.92%

Test set: Average loss: 0.5494, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.527578
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.537541
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.562313
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.531135
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.571579
Train Epoch: 22 - Loss: 0.5444, Accuracy: 98.93%

Test set: Average loss: 0.5490, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.525419
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.528446
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.540798
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.530240
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.545030
Train Epoch: 23 - Loss: 0.5442, Accuracy: 98.95%

Test set: Average loss: 0.5490, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.572279
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.565296
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.542488
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.541223
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.539143
Train Epoch: 24 - Loss: 0.5441, Accuracy: 98.95%

Test set: Average loss: 0.5490, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.535259
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.527203
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.557055
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.547871
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.530891
Train Epoch: 25 - Loss: 0.5440, Accuracy: 98.96%

Test set: Average loss: 0.5489, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.552494
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.529243
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.578112
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.549273
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.550173
Train Epoch: 26 - Loss: 0.5466, Accuracy: 98.85%

Test set: Average loss: 0.5537, Accuracy: 9851/10000 (98.51%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.525392
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.552532
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.534909
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.539142
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.519845
Train Epoch: 27 - Loss: 0.5444, Accuracy: 98.92%

Test set: Average loss: 0.5474, Accuracy: 9853/10000 (98.53%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.526121
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.546866
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.575820
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.532041
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.515426
Train Epoch: 28 - Loss: 0.5423, Accuracy: 99.00%

Test set: Average loss: 0.5460, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.530773
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.537120
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.546134
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.538381
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.532801
Train Epoch: 29 - Loss: 0.5408, Accuracy: 99.01%

Test set: Average loss: 0.5451, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.524152
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.578422
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.522329
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.532851
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.542890
Train Epoch: 30 - Loss: 0.5391, Accuracy: 99.03%

Test set: Average loss: 0.5445, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.524127
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.550892
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.529065
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.536708
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.571234
Train Epoch: 31 - Loss: 0.5378, Accuracy: 99.12%

Test set: Average loss: 0.5434, Accuracy: 9867/10000 (98.67%)

New best model saved with accuracy: 98.67%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.515089
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.524171
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.560416
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.516075
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.535181
Train Epoch: 32 - Loss: 0.5366, Accuracy: 99.12%

Test set: Average loss: 0.5428, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.539686
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.546558
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.521148
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.527513
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.530954
Train Epoch: 33 - Loss: 0.5354, Accuracy: 99.16%

Test set: Average loss: 0.5409, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.545240
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.523406
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.552737
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.541790
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.522283
Train Epoch: 34 - Loss: 0.5342, Accuracy: 99.24%

Test set: Average loss: 0.5408, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.522947
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.522181
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.520490
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.537134
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.533168
Train Epoch: 35 - Loss: 0.5333, Accuracy: 99.27%

Test set: Average loss: 0.5402, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.527177
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.521770
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.533441
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.534806
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.532007
Train Epoch: 36 - Loss: 0.5325, Accuracy: 99.26%

Test set: Average loss: 0.5394, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.525458
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.519740
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.524917
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.531401
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.516322
Train Epoch: 37 - Loss: 0.5317, Accuracy: 99.30%

Test set: Average loss: 0.5392, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.541465
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.515271
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.534915
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.545293
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.516259
Train Epoch: 38 - Loss: 0.5310, Accuracy: 99.33%

Test set: Average loss: 0.5385, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.528512
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.537594
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.526923
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.525775
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.543003
Train Epoch: 39 - Loss: 0.5304, Accuracy: 99.35%

Test set: Average loss: 0.5385, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.525975
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.541428
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.539645
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.563646
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.528067
Train Epoch: 40 - Loss: 0.5298, Accuracy: 99.38%

Test set: Average loss: 0.5381, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.522108
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.524476
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.531471
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.526409
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.513646
Train Epoch: 41 - Loss: 0.5294, Accuracy: 99.39%

Test set: Average loss: 0.5384, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.525596
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.535658
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.516457
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.524903
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.525930
Train Epoch: 42 - Loss: 0.5291, Accuracy: 99.39%

Test set: Average loss: 0.5377, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.546364
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.566785
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.525449
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.528612
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.532017
Train Epoch: 43 - Loss: 0.5286, Accuracy: 99.42%

Test set: Average loss: 0.5373, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.538118
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.518558
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.533908
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.514866
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.545752
Train Epoch: 44 - Loss: 0.5284, Accuracy: 99.42%

Test set: Average loss: 0.5370, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.523196
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.516795
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.527466
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.531217
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.549519
Train Epoch: 45 - Loss: 0.5281, Accuracy: 99.42%

Test set: Average loss: 0.5370, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.542033
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.523314
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.524895
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.524237
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.519890
Train Epoch: 46 - Loss: 0.5280, Accuracy: 99.43%

Test set: Average loss: 0.5369, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.521186
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.519460
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.543963
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.528139
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.535261
Train Epoch: 47 - Loss: 0.5278, Accuracy: 99.43%

Test set: Average loss: 0.5367, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.519112
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.526680
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.526222
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.518818
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.517395
Train Epoch: 48 - Loss: 0.5277, Accuracy: 99.43%

Test set: Average loss: 0.5368, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.518416
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.521901
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.538181
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.536305
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.513785
Train Epoch: 49 - Loss: 0.5276, Accuracy: 99.44%

Test set: Average loss: 0.5368, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.523904
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.522893
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.522667
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.535166
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.519780
Train Epoch: 50 - Loss: 0.5276, Accuracy: 99.44%

Test set: Average loss: 0.5367, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.510878
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.519473
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.525408
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.521057
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.556915
Train Epoch: 51 - Loss: 0.5296, Accuracy: 99.36%

Test set: Average loss: 0.5381, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.525557
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.540800
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.515481
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.519333
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.511021
Train Epoch: 52 - Loss: 0.5289, Accuracy: 99.39%

Test set: Average loss: 0.5397, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.527683
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.536269
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.553899
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.527384
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.515815
Train Epoch: 53 - Loss: 0.5281, Accuracy: 99.41%

Test set: Average loss: 0.5386, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.512849
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.525866
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.515230
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.522564
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.534872
Train Epoch: 54 - Loss: 0.5274, Accuracy: 99.42%

Test set: Average loss: 0.5352, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.522073
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.540036
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.521017
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.516935
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.515969
Train Epoch: 55 - Loss: 0.5267, Accuracy: 99.46%

Test set: Average loss: 0.5349, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.529518
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.545251
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514032
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.523713
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.514599
Train Epoch: 56 - Loss: 0.5260, Accuracy: 99.47%

Test set: Average loss: 0.5358, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.543708
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.522049
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.524198
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.534893
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.519965
Train Epoch: 57 - Loss: 0.5253, Accuracy: 99.46%

Test set: Average loss: 0.5363, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.523011
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.526862
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.520661
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.533332
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.522633
Train Epoch: 58 - Loss: 0.5247, Accuracy: 99.50%

Test set: Average loss: 0.5335, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.523651
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.525339
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.521090
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.529846
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.531146
Train Epoch: 59 - Loss: 0.5241, Accuracy: 99.52%

Test set: Average loss: 0.5340, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.509620
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.530357
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.533098
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.520484
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.540306
Train Epoch: 60 - Loss: 0.5236, Accuracy: 99.51%

Test set: Average loss: 0.5337, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.530620
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.520322
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.511810
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.527445
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.517972
Train Epoch: 61 - Loss: 0.5231, Accuracy: 99.54%

Test set: Average loss: 0.5343, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.516229
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.513094
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.519146
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.534006
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.519680
Train Epoch: 62 - Loss: 0.5227, Accuracy: 99.56%

Test set: Average loss: 0.5333, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.521277
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.511316
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.516096
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.526290
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.525381
Train Epoch: 63 - Loss: 0.5223, Accuracy: 99.56%

Test set: Average loss: 0.5326, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.515984
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.511570
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.527204
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.527387
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.515726
Train Epoch: 64 - Loss: 0.5219, Accuracy: 99.58%

Test set: Average loss: 0.5333, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.514445
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.522085
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.518514
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.513646
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.524172
Train Epoch: 65 - Loss: 0.5215, Accuracy: 99.60%

Test set: Average loss: 0.5325, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.514634
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.517619
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.513822
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.516938
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.515742
Train Epoch: 66 - Loss: 0.5212, Accuracy: 99.59%

Test set: Average loss: 0.5322, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.520071
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.517751
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.520016
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.523598
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.509998
Train Epoch: 67 - Loss: 0.5210, Accuracy: 99.61%

Test set: Average loss: 0.5322, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.519604
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.528574
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.516421
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.529172
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.539785
Train Epoch: 68 - Loss: 0.5208, Accuracy: 99.61%

Test set: Average loss: 0.5322, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.520175
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.522081
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.515109
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.518726
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.508436
Train Epoch: 69 - Loss: 0.5206, Accuracy: 99.63%

Test set: Average loss: 0.5320, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.517900
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.518505
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.514742
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.528278
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.514241
Train Epoch: 70 - Loss: 0.5204, Accuracy: 99.61%

Test set: Average loss: 0.5319, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.513240
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.516818
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.522035
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.535253
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.539946
Train Epoch: 71 - Loss: 0.5203, Accuracy: 99.63%

Test set: Average loss: 0.5318, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.513429
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.519273
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.512997
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.514303
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.508087
Train Epoch: 72 - Loss: 0.5202, Accuracy: 99.63%

Test set: Average loss: 0.5319, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.513087
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.525203
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.522642
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.548960
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.512812
Train Epoch: 73 - Loss: 0.5201, Accuracy: 99.64%

Test set: Average loss: 0.5317, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.526750
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.523741
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.525840
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.511860
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513299
Train Epoch: 74 - Loss: 0.5201, Accuracy: 99.64%

Test set: Average loss: 0.5317, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.511295
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.509234
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.527852
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.513206
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.524752
Train Epoch: 75 - Loss: 0.5200, Accuracy: 99.64%

Test set: Average loss: 0.5317, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.512523
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.515240
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.524037
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.519919
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.524086
Train Epoch: 76 - Loss: 0.5217, Accuracy: 99.57%

Test set: Average loss: 0.5323, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.513995
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.513736
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.511457
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.527308
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.521014
Train Epoch: 77 - Loss: 0.5212, Accuracy: 99.61%

Test set: Average loss: 0.5320, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.534085
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.514955
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.514568
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.518857
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.529404
Train Epoch: 78 - Loss: 0.5209, Accuracy: 99.60%

Test set: Average loss: 0.5316, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.518108
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.546061
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.516747
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.517055
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.522354
Train Epoch: 79 - Loss: 0.5204, Accuracy: 99.61%

Test set: Average loss: 0.5316, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.511140
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.517454
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.513650
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.519509
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.522150
Train Epoch: 80 - Loss: 0.5199, Accuracy: 99.62%

Test set: Average loss: 0.5312, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.513802
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.513855
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.537350
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.522417
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.514198
Train Epoch: 81 - Loss: 0.5195, Accuracy: 99.64%

Test set: Average loss: 0.5314, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.516199
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.527458
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.517855
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.514173
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.517313
Train Epoch: 82 - Loss: 0.5190, Accuracy: 99.65%

Test set: Average loss: 0.5314, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.560536
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.510292
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.521247
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.523630
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.517491
Train Epoch: 83 - Loss: 0.5187, Accuracy: 99.66%

Test set: Average loss: 0.5309, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.514778
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.531738
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.514747
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.517240
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.516136
Train Epoch: 84 - Loss: 0.5183, Accuracy: 99.66%

Test set: Average loss: 0.5317, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.513232
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.523606
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.514092
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.519436
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.512547
Train Epoch: 85 - Loss: 0.5179, Accuracy: 99.67%

Test set: Average loss: 0.5301, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.509338
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.510734
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.524169
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.511174
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.512357
Train Epoch: 86 - Loss: 0.5176, Accuracy: 99.69%

Test set: Average loss: 0.5308, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.515415
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.519940
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.511967
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.509912
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.510912
Train Epoch: 87 - Loss: 0.5173, Accuracy: 99.70%

Test set: Average loss: 0.5304, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.529249
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.511553
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.510840
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.512511
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.521118
Train Epoch: 88 - Loss: 0.5170, Accuracy: 99.70%

Test set: Average loss: 0.5298, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.515862
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.512889
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.518650
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.519002
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.521014
Train Epoch: 89 - Loss: 0.5167, Accuracy: 99.72%

Test set: Average loss: 0.5301, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.511787
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.526155
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.513596
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.516338
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.528831
Train Epoch: 90 - Loss: 0.5165, Accuracy: 99.72%

Test set: Average loss: 0.5295, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.510454
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.511877
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.513590
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.523620
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.526521
Train Epoch: 91 - Loss: 0.5163, Accuracy: 99.73%

Test set: Average loss: 0.5294, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.516353
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.524160
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.531859
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.518325
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.511784
Train Epoch: 92 - Loss: 0.5161, Accuracy: 99.74%

Test set: Average loss: 0.5294, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.520150
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.519666
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.514366
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.511058
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.513859
Train Epoch: 93 - Loss: 0.5159, Accuracy: 99.73%

Test set: Average loss: 0.5292, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.525119
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509359
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.508958
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.508007
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.514617
Train Epoch: 94 - Loss: 0.5158, Accuracy: 99.74%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.524215
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.512065
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.512129
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.513650
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.513917
Train Epoch: 95 - Loss: 0.5157, Accuracy: 99.75%

Test set: Average loss: 0.5294, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.528344
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.521119
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.509274
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.512386
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.510463
Train Epoch: 96 - Loss: 0.5156, Accuracy: 99.75%

Test set: Average loss: 0.5293, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.516208
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509257
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.509313
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.506990
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.518803
Train Epoch: 97 - Loss: 0.5155, Accuracy: 99.75%

Test set: Average loss: 0.5291, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.538476
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.506992
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.507239
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511502
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.528898
Train Epoch: 98 - Loss: 0.5155, Accuracy: 99.75%

Test set: Average loss: 0.5291, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.517566
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.511283
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.519859
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.512195
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.519370
Train Epoch: 99 - Loss: 0.5154, Accuracy: 99.76%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.507482
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.513271
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.510525
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.510247
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.521803
Train Epoch: 100 - Loss: 0.5154, Accuracy: 99.76%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)


Training completed. Best Accuracy: 99.04%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.307731
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.233346
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.906058
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.864112
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.791373
Train Epoch: 1 - Loss: 1.1223, Accuracy: 77.96%

Test set: Average loss: 0.7642, Accuracy: 9145/10000 (91.45%)

New best model saved with accuracy: 91.45%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.761819
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.735651
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.710510
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.685297
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.700678
Train Epoch: 2 - Loss: 0.7166, Accuracy: 93.32%

Test set: Average loss: 0.6640, Accuracy: 9514/10000 (95.14%)

New best model saved with accuracy: 95.14%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.632490
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.679040
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.658351
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.673677
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.626913
Train Epoch: 3 - Loss: 0.6471, Accuracy: 95.72%

Test set: Average loss: 0.6199, Accuracy: 9672/10000 (96.72%)

New best model saved with accuracy: 96.72%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.637138
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.657420
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.621082
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.604324
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.596962
Train Epoch: 4 - Loss: 0.6138, Accuracy: 96.83%

Test set: Average loss: 0.5947, Accuracy: 9735/10000 (97.35%)

New best model saved with accuracy: 97.35%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.663900
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.562320
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.604773
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.597377
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.578580
Train Epoch: 5 - Loss: 0.5956, Accuracy: 97.37%

Test set: Average loss: 0.5841, Accuracy: 9764/10000 (97.64%)

New best model saved with accuracy: 97.64%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.640689
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.568209
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.587901
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.603614
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.576632
Train Epoch: 6 - Loss: 0.5839, Accuracy: 97.74%

Test set: Average loss: 0.5811, Accuracy: 9777/10000 (97.77%)

New best model saved with accuracy: 97.77%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.561519
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.582402
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.571020
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.643110
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.619332
Train Epoch: 7 - Loss: 0.5757, Accuracy: 97.96%

Test set: Average loss: 0.5691, Accuracy: 9796/10000 (97.96%)

New best model saved with accuracy: 97.96%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.551014
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.569508
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.571667
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.601088
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.564094
Train Epoch: 8 - Loss: 0.5694, Accuracy: 98.20%

Test set: Average loss: 0.5682, Accuracy: 9803/10000 (98.03%)

New best model saved with accuracy: 98.03%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.573991
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.576707
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.540664
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.572945
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.587890
Train Epoch: 9 - Loss: 0.5647, Accuracy: 98.33%

Test set: Average loss: 0.5624, Accuracy: 9826/10000 (98.26%)

New best model saved with accuracy: 98.26%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.548691
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.600744
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.536247
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.541748
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.547975
Train Epoch: 10 - Loss: 0.5608, Accuracy: 98.43%

Test set: Average loss: 0.5585, Accuracy: 9831/10000 (98.31%)

New best model saved with accuracy: 98.31%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.546753
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.556126
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.538997
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.540375
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.551120
Train Epoch: 11 - Loss: 0.5578, Accuracy: 98.52%

Test set: Average loss: 0.5575, Accuracy: 9831/10000 (98.31%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.547295
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.534572
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.566828
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.573313
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.546762
Train Epoch: 12 - Loss: 0.5552, Accuracy: 98.58%

Test set: Average loss: 0.5560, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.602698
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.538767
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.585534
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.568119
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.534452
Train Epoch: 13 - Loss: 0.5531, Accuracy: 98.72%

Test set: Average loss: 0.5539, Accuracy: 9846/10000 (98.46%)

New best model saved with accuracy: 98.46%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.553835
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.544534
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.559729
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.562269
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.547941
Train Epoch: 14 - Loss: 0.5512, Accuracy: 98.74%

Test set: Average loss: 0.5524, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.551984
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.545014
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.531387
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.530192
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.545357
Train Epoch: 15 - Loss: 0.5498, Accuracy: 98.75%

Test set: Average loss: 0.5512, Accuracy: 9849/10000 (98.49%)

New best model saved with accuracy: 98.49%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.553816
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.561131
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.552866
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.560629
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.532370
Train Epoch: 16 - Loss: 0.5486, Accuracy: 98.81%

Test set: Average loss: 0.5506, Accuracy: 9846/10000 (98.46%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.580532
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.566367
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.547817
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.549855
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.556363
Train Epoch: 17 - Loss: 0.5475, Accuracy: 98.83%

Test set: Average loss: 0.5501, Accuracy: 9851/10000 (98.51%)

New best model saved with accuracy: 98.51%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.527444
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.554273
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.543070
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.533961
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.547798
Train Epoch: 18 - Loss: 0.5467, Accuracy: 98.85%

Test set: Average loss: 0.5500, Accuracy: 9852/10000 (98.52%)

New best model saved with accuracy: 98.52%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.527767
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.552441
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.535405
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.532449
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.586668
Train Epoch: 19 - Loss: 0.5460, Accuracy: 98.89%

Test set: Average loss: 0.5488, Accuracy: 9857/10000 (98.57%)

New best model saved with accuracy: 98.57%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.540942
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.529212
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.542497
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.524204
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.533799
Train Epoch: 20 - Loss: 0.5455, Accuracy: 98.90%

Test set: Average loss: 0.5485, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.545984
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.544941
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.526845
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.541817
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.555923
Train Epoch: 21 - Loss: 0.5451, Accuracy: 98.91%

Test set: Average loss: 0.5484, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.545974
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.556941
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.550823
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.549999
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.530237
Train Epoch: 22 - Loss: 0.5448, Accuracy: 98.92%

Test set: Average loss: 0.5483, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.540533
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.528739
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.542691
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.539326
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.563377
Train Epoch: 23 - Loss: 0.5446, Accuracy: 98.93%

Test set: Average loss: 0.5481, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.541925
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.537399
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.547433
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.551053
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.541859
Train Epoch: 24 - Loss: 0.5445, Accuracy: 98.93%

Test set: Average loss: 0.5480, Accuracy: 9858/10000 (98.58%)

New best model saved with accuracy: 98.58%
Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.540547
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.563355
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.548302
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.575734
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.537580
Train Epoch: 25 - Loss: 0.5444, Accuracy: 98.93%

Test set: Average loss: 0.5480, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.535009
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.551804
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.558108
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.555003
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.555721
Train Epoch: 26 - Loss: 0.5472, Accuracy: 98.84%

Test set: Average loss: 0.5485, Accuracy: 9851/10000 (98.51%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.577894
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.550362
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.561571
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.525392
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.549923
Train Epoch: 27 - Loss: 0.5449, Accuracy: 98.90%

Test set: Average loss: 0.5491, Accuracy: 9845/10000 (98.45%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.547043
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.535248
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.535418
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.552899
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.521152
Train Epoch: 28 - Loss: 0.5429, Accuracy: 98.96%

Test set: Average loss: 0.5453, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.558277
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.550647
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.551126
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.522144
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.536534
Train Epoch: 29 - Loss: 0.5409, Accuracy: 99.02%

Test set: Average loss: 0.5452, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.539916
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.534349
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.539181
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.524577
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.551444
Train Epoch: 30 - Loss: 0.5394, Accuracy: 99.03%

Test set: Average loss: 0.5439, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.532841
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.536879
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.538993
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.522330
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.555145
Train Epoch: 31 - Loss: 0.5380, Accuracy: 99.09%

Test set: Average loss: 0.5418, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.521125
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.535702
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.556238
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.552795
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.533625
Train Epoch: 32 - Loss: 0.5367, Accuracy: 99.11%

Test set: Average loss: 0.5416, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.522761
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.523395
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.549595
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.549472
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.527117
Train Epoch: 33 - Loss: 0.5355, Accuracy: 99.14%

Test set: Average loss: 0.5393, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.538126
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.523898
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.522720
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.537283
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.536567
Train Epoch: 34 - Loss: 0.5345, Accuracy: 99.18%

Test set: Average loss: 0.5393, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.555919
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.548490
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.537968
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.514079
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.528177
Train Epoch: 35 - Loss: 0.5334, Accuracy: 99.22%

Test set: Average loss: 0.5397, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.535152
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.543326
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.558302
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.551587
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.554204
Train Epoch: 36 - Loss: 0.5326, Accuracy: 99.23%

Test set: Average loss: 0.5376, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.533115
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.519349
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.519893
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.531681
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.541263
Train Epoch: 37 - Loss: 0.5317, Accuracy: 99.28%

Test set: Average loss: 0.5377, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.536968
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.519591
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.544381
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.526649
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.529940
Train Epoch: 38 - Loss: 0.5310, Accuracy: 99.29%

Test set: Average loss: 0.5371, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.522909
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.558449
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.545470
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.518163
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.540813
Train Epoch: 39 - Loss: 0.5305, Accuracy: 99.32%

Test set: Average loss: 0.5372, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.516012
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.574499
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.541533
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.519487
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.532056
Train Epoch: 40 - Loss: 0.5299, Accuracy: 99.32%

Test set: Average loss: 0.5365, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.532121
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.530075
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.541178
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.526813
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.524448
Train Epoch: 41 - Loss: 0.5294, Accuracy: 99.35%

Test set: Average loss: 0.5359, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.521519
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.523117
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.526381
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.542124
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.551499
Train Epoch: 42 - Loss: 0.5290, Accuracy: 99.36%

Test set: Average loss: 0.5355, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.556526
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.527653
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.520856
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.522424
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.534557
Train Epoch: 43 - Loss: 0.5287, Accuracy: 99.37%

Test set: Average loss: 0.5355, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.536046
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.534000
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.539168
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.535716
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.517042
Train Epoch: 44 - Loss: 0.5283, Accuracy: 99.39%

Test set: Average loss: 0.5352, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.531664
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.526929
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.516338
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.542818
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.539981
Train Epoch: 45 - Loss: 0.5281, Accuracy: 99.40%

Test set: Average loss: 0.5351, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.524745
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.525527
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.521457
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.550873
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.536275
Train Epoch: 46 - Loss: 0.5279, Accuracy: 99.41%

Test set: Average loss: 0.5352, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.522413
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.519217
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.532218
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.512170
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.526469
Train Epoch: 47 - Loss: 0.5278, Accuracy: 99.41%

Test set: Average loss: 0.5352, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.534471
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.520184
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.545787
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.529730
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.527117
Train Epoch: 48 - Loss: 0.5277, Accuracy: 99.42%

Test set: Average loss: 0.5350, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.558674
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.538656
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.527724
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.545700
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.532828
Train Epoch: 49 - Loss: 0.5276, Accuracy: 99.42%

Test set: Average loss: 0.5350, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.514741
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.540814
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.516625
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.522985
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.531619
Train Epoch: 50 - Loss: 0.5275, Accuracy: 99.42%

Test set: Average loss: 0.5350, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.535032
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.553155
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.526866
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.521451
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.529775
Train Epoch: 51 - Loss: 0.5295, Accuracy: 99.34%

Test set: Average loss: 0.5362, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.580265
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.528043
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.530072
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.516566
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.536828
Train Epoch: 52 - Loss: 0.5289, Accuracy: 99.36%

Test set: Average loss: 0.5364, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.518871
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.548310
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.518290
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.534643
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.547409
Train Epoch: 53 - Loss: 0.5280, Accuracy: 99.39%

Test set: Average loss: 0.5346, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.515618
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.521510
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.523106
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.525699
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.517234
Train Epoch: 54 - Loss: 0.5272, Accuracy: 99.39%

Test set: Average loss: 0.5345, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.530206
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.531468
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.531126
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.535751
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.515706
Train Epoch: 55 - Loss: 0.5265, Accuracy: 99.41%

Test set: Average loss: 0.5339, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.520974
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.527840
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.547944
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.540450
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.526175
Train Epoch: 56 - Loss: 0.5258, Accuracy: 99.42%

Test set: Average loss: 0.5325, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.514600
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.512044
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.533608
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.524072
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.526864
Train Epoch: 57 - Loss: 0.5251, Accuracy: 99.46%

Test set: Average loss: 0.5344, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.516593
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.513044
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.540213
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.520540
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.533572
Train Epoch: 58 - Loss: 0.5244, Accuracy: 99.48%

Test set: Average loss: 0.5326, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.514723
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.518856
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.531416
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.512955
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.527407
Train Epoch: 59 - Loss: 0.5239, Accuracy: 99.50%

Test set: Average loss: 0.5316, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.524823
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.512629
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.508732
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.527615
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.513403
Train Epoch: 60 - Loss: 0.5234, Accuracy: 99.52%

Test set: Average loss: 0.5320, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.525737
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.514489
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.527409
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.529514
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.514328
Train Epoch: 61 - Loss: 0.5229, Accuracy: 99.53%

Test set: Average loss: 0.5308, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.510578
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.514721
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.509124
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.523380
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.518881
Train Epoch: 62 - Loss: 0.5225, Accuracy: 99.53%

Test set: Average loss: 0.5311, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.525266
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.519842
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.512795
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.514138
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.519129
Train Epoch: 63 - Loss: 0.5220, Accuracy: 99.55%

Test set: Average loss: 0.5311, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.518968
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.520895
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.541437
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.513666
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.515583
Train Epoch: 64 - Loss: 0.5217, Accuracy: 99.56%

Test set: Average loss: 0.5303, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.513605
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.528960
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.524817
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.510041
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.508656
Train Epoch: 65 - Loss: 0.5213, Accuracy: 99.56%

Test set: Average loss: 0.5307, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.514882
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.516621
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.518324
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.522669
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.516909
Train Epoch: 66 - Loss: 0.5210, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.524534
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.532263
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.538687
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.512152
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513336
Train Epoch: 67 - Loss: 0.5208, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.516413
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.512512
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.511015
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.518768
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.510201
Train Epoch: 68 - Loss: 0.5205, Accuracy: 99.61%

Test set: Average loss: 0.5294, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.513631
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.517562
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.519827
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.547240
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.516469
Train Epoch: 69 - Loss: 0.5204, Accuracy: 99.61%

Test set: Average loss: 0.5296, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.518946
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.514277
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.529903
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.545276
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.526029
Train Epoch: 70 - Loss: 0.5202, Accuracy: 99.60%

Test set: Average loss: 0.5293, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.523658
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.516397
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.510334
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.517643
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.548187
Train Epoch: 71 - Loss: 0.5201, Accuracy: 99.61%

Test set: Average loss: 0.5293, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.516093
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.522193
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.515207
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.509975
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.519554
Train Epoch: 72 - Loss: 0.5200, Accuracy: 99.63%

Test set: Average loss: 0.5293, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.510390
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.525405
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.523312
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.523315
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.514074
Train Epoch: 73 - Loss: 0.5199, Accuracy: 99.63%

Test set: Average loss: 0.5294, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.534773
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.521005
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.515837
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.535438
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.527614
Train Epoch: 74 - Loss: 0.5199, Accuracy: 99.63%

Test set: Average loss: 0.5293, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.560393
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.513644
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.521308
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.522942
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.516055
Train Epoch: 75 - Loss: 0.5198, Accuracy: 99.64%

Test set: Average loss: 0.5293, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.521851
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.515945
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.525029
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.513127
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.526190
Train Epoch: 76 - Loss: 0.5215, Accuracy: 99.56%

Test set: Average loss: 0.5297, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.509409
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.518791
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.543122
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.511113
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.514042
Train Epoch: 77 - Loss: 0.5212, Accuracy: 99.56%

Test set: Average loss: 0.5299, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.515593
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.526403
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.517328
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.513638
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.528243
Train Epoch: 78 - Loss: 0.5205, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.515332
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.509078
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.518054
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.516896
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.531176
Train Epoch: 79 - Loss: 0.5202, Accuracy: 99.58%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.512691
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.509544
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.511436
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.519822
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.515284
Train Epoch: 80 - Loss: 0.5197, Accuracy: 99.61%

Test set: Average loss: 0.5284, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.530318
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.528996
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.513744
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.509025
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.523204
Train Epoch: 81 - Loss: 0.5192, Accuracy: 99.64%

Test set: Average loss: 0.5285, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.516148
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.517082
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.511130
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.511681
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.523505
Train Epoch: 82 - Loss: 0.5189, Accuracy: 99.64%

Test set: Average loss: 0.5288, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.509941
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.523384
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.513464
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.525978
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.510134
Train Epoch: 83 - Loss: 0.5184, Accuracy: 99.66%

Test set: Average loss: 0.5279, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.520833
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.508493
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.513060
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.508756
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.509510
Train Epoch: 84 - Loss: 0.5181, Accuracy: 99.66%

Test set: Average loss: 0.5280, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.513937
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.510150
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.518600
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.527510
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.515727
Train Epoch: 85 - Loss: 0.5177, Accuracy: 99.68%

Test set: Average loss: 0.5275, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.510609
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.522500
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.514379
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.510984
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.563897
Train Epoch: 86 - Loss: 0.5174, Accuracy: 99.67%

Test set: Average loss: 0.5272, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.511506
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.527363
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.510108
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.512049
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.514721
Train Epoch: 87 - Loss: 0.5171, Accuracy: 99.69%

Test set: Average loss: 0.5270, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.517309
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.513105
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.508960
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.521750
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.506904
Train Epoch: 88 - Loss: 0.5168, Accuracy: 99.70%

Test set: Average loss: 0.5269, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.521151
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.528379
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.517211
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.522450
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.519187
Train Epoch: 89 - Loss: 0.5165, Accuracy: 99.70%

Test set: Average loss: 0.5267, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.511403
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.508750
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.506736
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.531627
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.513287
Train Epoch: 90 - Loss: 0.5163, Accuracy: 99.71%

Test set: Average loss: 0.5268, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.511654
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.516151
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.514727
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.513220
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.506194
Train Epoch: 91 - Loss: 0.5161, Accuracy: 99.70%

Test set: Average loss: 0.5267, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.508830
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.509273
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.522676
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.529990
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.524854
Train Epoch: 92 - Loss: 0.5159, Accuracy: 99.72%

Test set: Average loss: 0.5265, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.509026
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.507994
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.520323
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.518691
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.505138
Train Epoch: 93 - Loss: 0.5158, Accuracy: 99.74%

Test set: Average loss: 0.5265, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.518198
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.514404
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.510381
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.506240
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.512475
Train Epoch: 94 - Loss: 0.5156, Accuracy: 99.73%

Test set: Average loss: 0.5263, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.527686
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.512555
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.507073
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.513356
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.518207
Train Epoch: 95 - Loss: 0.5155, Accuracy: 99.73%

Test set: Average loss: 0.5265, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.507465
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.513155
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.509473
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.509238
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.522527
Train Epoch: 96 - Loss: 0.5154, Accuracy: 99.73%

Test set: Average loss: 0.5263, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.505754
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.519807
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.514136
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.521572
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.509845
Train Epoch: 97 - Loss: 0.5153, Accuracy: 99.73%

Test set: Average loss: 0.5264, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.519145
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.507355
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.514101
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508507
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.519490
Train Epoch: 98 - Loss: 0.5153, Accuracy: 99.74%

Test set: Average loss: 0.5262, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.530043
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.510089
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.510458
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.507719
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.519265
Train Epoch: 99 - Loss: 0.5152, Accuracy: 99.74%

Test set: Average loss: 0.5263, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.512581
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.514727
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.510644
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.509960
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.510625
Train Epoch: 100 - Loss: 0.5152, Accuracy: 99.75%

Test set: Average loss: 0.5263, Accuracy: 9913/10000 (99.13%)


Training completed. Best Accuracy: 99.14%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.309944
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.273042
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.975439
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.802073
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.765772
Train Epoch: 1 - Loss: 1.0984, Accuracy: 78.42%

Test set: Average loss: 0.7511, Accuracy: 9256/10000 (92.56%)

New best model saved with accuracy: 92.56%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.778342
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.743965
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.705764
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.707544
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.667836
Train Epoch: 2 - Loss: 0.7080, Accuracy: 93.65%

Test set: Average loss: 0.6564, Accuracy: 9536/10000 (95.36%)

New best model saved with accuracy: 95.36%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.628163
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.653828
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.710443
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.622452
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.606160
Train Epoch: 3 - Loss: 0.6437, Accuracy: 95.77%

Test set: Average loss: 0.6250, Accuracy: 9638/10000 (96.38%)

New best model saved with accuracy: 96.38%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.616951
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.610250
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.612177
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.582711
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.594506
Train Epoch: 4 - Loss: 0.6133, Accuracy: 96.77%

Test set: Average loss: 0.5979, Accuracy: 9716/10000 (97.16%)

New best model saved with accuracy: 97.16%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.618426
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.605713
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.585501
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.627169
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.591493
Train Epoch: 5 - Loss: 0.5958, Accuracy: 97.28%

Test set: Average loss: 0.5864, Accuracy: 9758/10000 (97.58%)

New best model saved with accuracy: 97.58%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.580656
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.570586
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.602612
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.560077
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.614333
Train Epoch: 6 - Loss: 0.5843, Accuracy: 97.64%

Test set: Average loss: 0.5747, Accuracy: 9796/10000 (97.96%)

New best model saved with accuracy: 97.96%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.580322
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.614105
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.591216
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.567473
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.560849
Train Epoch: 7 - Loss: 0.5761, Accuracy: 97.93%

Test set: Average loss: 0.5699, Accuracy: 9811/10000 (98.11%)

New best model saved with accuracy: 98.11%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.580200
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.594281
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.587525
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.566784
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.534529
Train Epoch: 8 - Loss: 0.5700, Accuracy: 98.10%

Test set: Average loss: 0.5654, Accuracy: 9811/10000 (98.11%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.565436
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.593676
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.528880
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.536566
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.535259
Train Epoch: 9 - Loss: 0.5651, Accuracy: 98.23%

Test set: Average loss: 0.5633, Accuracy: 9819/10000 (98.19%)

New best model saved with accuracy: 98.19%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.577510
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.552480
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.555076
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.567376
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.558925
Train Epoch: 10 - Loss: 0.5614, Accuracy: 98.36%

Test set: Average loss: 0.5631, Accuracy: 9819/10000 (98.19%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.555468
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.538349
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.545696
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.586192
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.588461
Train Epoch: 11 - Loss: 0.5582, Accuracy: 98.47%

Test set: Average loss: 0.5570, Accuracy: 9840/10000 (98.40%)

New best model saved with accuracy: 98.40%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.554269
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.533412
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.563214
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.546267
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.532691
Train Epoch: 12 - Loss: 0.5555, Accuracy: 98.56%

Test set: Average loss: 0.5572, Accuracy: 9846/10000 (98.46%)

New best model saved with accuracy: 98.46%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.565455
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.530396
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.549992
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.556392
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.533186
Train Epoch: 13 - Loss: 0.5533, Accuracy: 98.61%

Test set: Average loss: 0.5545, Accuracy: 9848/10000 (98.48%)

New best model saved with accuracy: 98.48%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.574993
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.538313
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.531802
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.547194
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.554262
Train Epoch: 14 - Loss: 0.5516, Accuracy: 98.67%

Test set: Average loss: 0.5533, Accuracy: 9847/10000 (98.47%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.546662
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.554575
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.525761
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.552624
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.566408
Train Epoch: 15 - Loss: 0.5500, Accuracy: 98.72%

Test set: Average loss: 0.5516, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.537454
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.533138
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.547068
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.539808
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.524948
Train Epoch: 16 - Loss: 0.5486, Accuracy: 98.77%

Test set: Average loss: 0.5508, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.562652
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.560096
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.550097
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.555424
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.557200
Train Epoch: 17 - Loss: 0.5477, Accuracy: 98.82%

Test set: Average loss: 0.5503, Accuracy: 9852/10000 (98.52%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.536630
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.546709
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.537586
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.564999
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.552466
Train Epoch: 18 - Loss: 0.5468, Accuracy: 98.86%

Test set: Average loss: 0.5492, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.539297
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.623378
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.530163
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.540746
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.551109
Train Epoch: 19 - Loss: 0.5461, Accuracy: 98.85%

Test set: Average loss: 0.5488, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.544526
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.534185
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.573584
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.548940
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.536581
Train Epoch: 20 - Loss: 0.5456, Accuracy: 98.89%

Test set: Average loss: 0.5484, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.556084
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.558825
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.537060
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.541261
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.539581
Train Epoch: 21 - Loss: 0.5452, Accuracy: 98.88%

Test set: Average loss: 0.5483, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.583423
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.546856
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.551973
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.535461
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.558367
Train Epoch: 22 - Loss: 0.5449, Accuracy: 98.88%

Test set: Average loss: 0.5481, Accuracy: 9862/10000 (98.62%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.543258
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.542437
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.566139
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.575520
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.562873
Train Epoch: 23 - Loss: 0.5447, Accuracy: 98.91%

Test set: Average loss: 0.5480, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.526889
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.555327
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.523125
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.523583
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.543708
Train Epoch: 24 - Loss: 0.5446, Accuracy: 98.91%

Test set: Average loss: 0.5480, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.543845
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.536107
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.532311
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.540441
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.528322
Train Epoch: 25 - Loss: 0.5445, Accuracy: 98.91%

Test set: Average loss: 0.5480, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.548935
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.588766
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.554293
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.538114
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.526794
Train Epoch: 26 - Loss: 0.5470, Accuracy: 98.78%

Test set: Average loss: 0.5517, Accuracy: 9854/10000 (98.54%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.551517
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.546826
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.534761
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.539091
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.573898
Train Epoch: 27 - Loss: 0.5449, Accuracy: 98.91%

Test set: Average loss: 0.5476, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.526347
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.551926
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.553502
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.530914
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.550050
Train Epoch: 28 - Loss: 0.5429, Accuracy: 98.91%

Test set: Average loss: 0.5455, Accuracy: 9866/10000 (98.66%)

New best model saved with accuracy: 98.66%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.537451
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.539436
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.535623
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.557824
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.531849
Train Epoch: 29 - Loss: 0.5410, Accuracy: 98.98%

Test set: Average loss: 0.5466, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.531331
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.552222
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.571646
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.562293
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.546368
Train Epoch: 30 - Loss: 0.5395, Accuracy: 99.01%

Test set: Average loss: 0.5420, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.518707
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.537513
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.562658
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.550268
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.545914
Train Epoch: 31 - Loss: 0.5379, Accuracy: 99.08%

Test set: Average loss: 0.5422, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.551368
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.549352
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.534448
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.527298
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.529874
Train Epoch: 32 - Loss: 0.5365, Accuracy: 99.11%

Test set: Average loss: 0.5408, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.530508
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.520654
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.527588
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.525186
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.528405
Train Epoch: 33 - Loss: 0.5353, Accuracy: 99.15%

Test set: Average loss: 0.5419, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.569212
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.526273
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.522639
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.533222
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.576929
Train Epoch: 34 - Loss: 0.5342, Accuracy: 99.18%

Test set: Average loss: 0.5395, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.524309
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.546682
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.525118
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.538742
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.534278
Train Epoch: 35 - Loss: 0.5333, Accuracy: 99.19%

Test set: Average loss: 0.5397, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.515656
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.566844
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.530086
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.512389
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.519367
Train Epoch: 36 - Loss: 0.5324, Accuracy: 99.24%

Test set: Average loss: 0.5379, Accuracy: 9877/10000 (98.77%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.530672
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.532320
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.535038
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.533882
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.540537
Train Epoch: 37 - Loss: 0.5315, Accuracy: 99.27%

Test set: Average loss: 0.5378, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.544290
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.519211
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.535065
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.564405
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.521607
Train Epoch: 38 - Loss: 0.5309, Accuracy: 99.27%

Test set: Average loss: 0.5370, Accuracy: 9883/10000 (98.83%)

New best model saved with accuracy: 98.83%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.535142
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.518540
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.518590
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.517527
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.541051
Train Epoch: 39 - Loss: 0.5302, Accuracy: 99.34%

Test set: Average loss: 0.5362, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.522725
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.531977
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.528921
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.530396
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.540509
Train Epoch: 40 - Loss: 0.5296, Accuracy: 99.32%

Test set: Average loss: 0.5361, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.528652
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.532687
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.555943
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.522890
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.520875
Train Epoch: 41 - Loss: 0.5291, Accuracy: 99.34%

Test set: Average loss: 0.5363, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.538848
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.529317
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.535101
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.523073
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.524351
Train Epoch: 42 - Loss: 0.5288, Accuracy: 99.35%

Test set: Average loss: 0.5353, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.527338
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.529055
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.515315
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.509662
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.518158
Train Epoch: 43 - Loss: 0.5284, Accuracy: 99.37%

Test set: Average loss: 0.5355, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.529236
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.524268
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.524832
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.524251
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.514294
Train Epoch: 44 - Loss: 0.5281, Accuracy: 99.37%

Test set: Average loss: 0.5353, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.556625
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.512506
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.548343
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.539317
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.535284
Train Epoch: 45 - Loss: 0.5279, Accuracy: 99.39%

Test set: Average loss: 0.5352, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.529338
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.534594
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.520872
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.543396
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.534064
Train Epoch: 46 - Loss: 0.5277, Accuracy: 99.38%

Test set: Average loss: 0.5348, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.528746
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.519155
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.516084
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.555555
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.511236
Train Epoch: 47 - Loss: 0.5275, Accuracy: 99.39%

Test set: Average loss: 0.5349, Accuracy: 9882/10000 (98.82%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.526466
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.521978
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.522169
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.526319
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.512760
Train Epoch: 48 - Loss: 0.5274, Accuracy: 99.40%

Test set: Average loss: 0.5348, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.519411
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.521675
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.521331
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.533097
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.531502
Train Epoch: 49 - Loss: 0.5273, Accuracy: 99.41%

Test set: Average loss: 0.5348, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.517056
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.524258
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.520215
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.523144
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.516031
Train Epoch: 50 - Loss: 0.5273, Accuracy: 99.40%

Test set: Average loss: 0.5348, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.535699
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.523752
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.536791
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.529395
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.542383
Train Epoch: 51 - Loss: 0.5294, Accuracy: 99.30%

Test set: Average loss: 0.5361, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.526336
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.520374
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.532150
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.536351
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.536575
Train Epoch: 52 - Loss: 0.5286, Accuracy: 99.33%

Test set: Average loss: 0.5348, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.539887
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.522483
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.514631
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.539064
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.534269
Train Epoch: 53 - Loss: 0.5278, Accuracy: 99.36%

Test set: Average loss: 0.5353, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.531582
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.520995
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.524006
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.542889
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.530179
Train Epoch: 54 - Loss: 0.5270, Accuracy: 99.38%

Test set: Average loss: 0.5349, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.533625
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.526915
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.515307
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.531891
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.520590
Train Epoch: 55 - Loss: 0.5262, Accuracy: 99.42%

Test set: Average loss: 0.5332, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.526513
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.523398
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.535625
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.553046
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.523941
Train Epoch: 56 - Loss: 0.5256, Accuracy: 99.44%

Test set: Average loss: 0.5340, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.525183
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.514004
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.525812
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.516313
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.529423
Train Epoch: 57 - Loss: 0.5249, Accuracy: 99.45%

Test set: Average loss: 0.5327, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.514214
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.511973
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.512310
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.537795
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.514851
Train Epoch: 58 - Loss: 0.5244, Accuracy: 99.45%

Test set: Average loss: 0.5322, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.509550
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.516632
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.541595
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.507610
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.517571
Train Epoch: 59 - Loss: 0.5237, Accuracy: 99.48%

Test set: Average loss: 0.5317, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.513886
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.518782
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.527272
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.519144
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.520824
Train Epoch: 60 - Loss: 0.5232, Accuracy: 99.52%

Test set: Average loss: 0.5315, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.523982
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.516116
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.518857
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.540443
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.513967
Train Epoch: 61 - Loss: 0.5227, Accuracy: 99.52%

Test set: Average loss: 0.5330, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.510407
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.520216
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.529730
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.518194
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.534268
Train Epoch: 62 - Loss: 0.5223, Accuracy: 99.53%

Test set: Average loss: 0.5311, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.529583
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.531672
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.514875
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.527316
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.515255
Train Epoch: 63 - Loss: 0.5219, Accuracy: 99.53%

Test set: Average loss: 0.5306, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.514062
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.540310
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.515189
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.513309
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.524165
Train Epoch: 64 - Loss: 0.5215, Accuracy: 99.55%

Test set: Average loss: 0.5305, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.520147
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.512650
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.531443
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.510175
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.523792
Train Epoch: 65 - Loss: 0.5213, Accuracy: 99.56%

Test set: Average loss: 0.5306, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.523200
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.532627
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.512631
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.509948
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.534533
Train Epoch: 66 - Loss: 0.5209, Accuracy: 99.58%

Test set: Average loss: 0.5298, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.527654
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.518252
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.517877
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.523773
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513418
Train Epoch: 67 - Loss: 0.5206, Accuracy: 99.59%

Test set: Average loss: 0.5299, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.534442
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.553876
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.510449
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.529361
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.522505
Train Epoch: 68 - Loss: 0.5204, Accuracy: 99.60%

Test set: Average loss: 0.5299, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.523058
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.520486
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.512213
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.521000
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.534220
Train Epoch: 69 - Loss: 0.5203, Accuracy: 99.60%

Test set: Average loss: 0.5295, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.537311
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.520494
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.522482
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.519734
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.519831
Train Epoch: 70 - Loss: 0.5201, Accuracy: 99.61%

Test set: Average loss: 0.5297, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.515303
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.523174
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.525510
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.517632
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.531022
Train Epoch: 71 - Loss: 0.5200, Accuracy: 99.61%

Test set: Average loss: 0.5295, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.539493
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.516621
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509076
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512262
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.512872
Train Epoch: 72 - Loss: 0.5199, Accuracy: 99.62%

Test set: Average loss: 0.5294, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.510798
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.540776
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.521176
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.537996
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.530686
Train Epoch: 73 - Loss: 0.5198, Accuracy: 99.62%

Test set: Average loss: 0.5294, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.510916
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.523783
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.515670
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.529514
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.537807
Train Epoch: 74 - Loss: 0.5197, Accuracy: 99.62%

Test set: Average loss: 0.5294, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.514143
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.535592
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.517712
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.508770
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.523503
Train Epoch: 75 - Loss: 0.5197, Accuracy: 99.62%

Test set: Average loss: 0.5294, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.514343
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.510134
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.506727
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.524988
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.509691
Train Epoch: 76 - Loss: 0.5213, Accuracy: 99.56%

Test set: Average loss: 0.5297, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.510977
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.520084
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.517186
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.528584
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.519417
Train Epoch: 77 - Loss: 0.5209, Accuracy: 99.54%

Test set: Average loss: 0.5298, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.536636
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.527699
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.517886
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.518370
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.519473
Train Epoch: 78 - Loss: 0.5206, Accuracy: 99.57%

Test set: Average loss: 0.5307, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.519804
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.507953
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.528753
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.520543
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.517977
Train Epoch: 79 - Loss: 0.5200, Accuracy: 99.59%

Test set: Average loss: 0.5286, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.520663
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.512707
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.529624
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.511582
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.518360
Train Epoch: 80 - Loss: 0.5195, Accuracy: 99.61%

Test set: Average loss: 0.5288, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.516777
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.511027
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.527666
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.518453
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.513571
Train Epoch: 81 - Loss: 0.5192, Accuracy: 99.62%

Test set: Average loss: 0.5284, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.511437
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.514993
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.510000
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.525938
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.508895
Train Epoch: 82 - Loss: 0.5187, Accuracy: 99.64%

Test set: Average loss: 0.5292, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.518981
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.506741
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.530712
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.521640
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.523888
Train Epoch: 83 - Loss: 0.5184, Accuracy: 99.63%

Test set: Average loss: 0.5282, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.509803
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.520047
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.513053
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.510374
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.513239
Train Epoch: 84 - Loss: 0.5179, Accuracy: 99.66%

Test set: Average loss: 0.5284, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.517683
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.523752
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.513414
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.511855
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.519307
Train Epoch: 85 - Loss: 0.5176, Accuracy: 99.68%

Test set: Average loss: 0.5276, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.513071
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.514307
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.534004
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.551068
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.520160
Train Epoch: 86 - Loss: 0.5173, Accuracy: 99.67%

Test set: Average loss: 0.5279, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.509304
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.514292
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.517335
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.524470
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.505620
Train Epoch: 87 - Loss: 0.5170, Accuracy: 99.69%

Test set: Average loss: 0.5283, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.528905
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.520801
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.514045
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.509836
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.523678
Train Epoch: 88 - Loss: 0.5167, Accuracy: 99.70%

Test set: Average loss: 0.5274, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.515977
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.514969
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.514516
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.513950
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.512066
Train Epoch: 89 - Loss: 0.5165, Accuracy: 99.69%

Test set: Average loss: 0.5274, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.511603
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513808
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.517762
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.510312
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.516532
Train Epoch: 90 - Loss: 0.5163, Accuracy: 99.71%

Test set: Average loss: 0.5268, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.512197
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.512044
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.510265
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.511273
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.513412
Train Epoch: 91 - Loss: 0.5161, Accuracy: 99.71%

Test set: Average loss: 0.5268, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.527234
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.512713
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.515610
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.519702
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.517989
Train Epoch: 92 - Loss: 0.5159, Accuracy: 99.72%

Test set: Average loss: 0.5268, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.524217
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.508574
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.521836
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.537822
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.518423
Train Epoch: 93 - Loss: 0.5157, Accuracy: 99.72%

Test set: Average loss: 0.5266, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.507964
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.507325
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.514158
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.512819
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.515713
Train Epoch: 94 - Loss: 0.5156, Accuracy: 99.72%

Test set: Average loss: 0.5268, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.509094
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.514096
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.522873
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.522328
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.520637
Train Epoch: 95 - Loss: 0.5155, Accuracy: 99.74%

Test set: Average loss: 0.5267, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.510419
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.515825
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507900
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.519229
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.511124
Train Epoch: 96 - Loss: 0.5154, Accuracy: 99.74%

Test set: Average loss: 0.5266, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.518247
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.523328
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.509471
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.513643
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.523297
Train Epoch: 97 - Loss: 0.5153, Accuracy: 99.74%

Test set: Average loss: 0.5266, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506623
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.531619
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.507068
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.509544
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.529929
Train Epoch: 98 - Loss: 0.5152, Accuracy: 99.74%

Test set: Average loss: 0.5267, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.519285
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.506256
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.509157
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.526739
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.531461
Train Epoch: 99 - Loss: 0.5152, Accuracy: 99.75%

Test set: Average loss: 0.5266, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.504967
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.510432
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.512919
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.515633
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.513445
Train Epoch: 100 - Loss: 0.5152, Accuracy: 99.75%

Test set: Average loss: 0.5266, Accuracy: 9904/10000 (99.04%)


Training completed. Best Accuracy: 99.09%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.319426
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.332101
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.999283
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.882097
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.833621
Train Epoch: 1 - Loss: 1.1488, Accuracy: 76.27%

Test set: Average loss: 0.7636, Accuracy: 9182/10000 (91.82%)

New best model saved with accuracy: 91.82%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.756810
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.767908
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.770989
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.684128
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.744096
Train Epoch: 2 - Loss: 0.7214, Accuracy: 93.08%

Test set: Average loss: 0.6661, Accuracy: 9497/10000 (94.97%)

New best model saved with accuracy: 94.97%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.652138
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.663446
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.647661
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.638672
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.629393
Train Epoch: 3 - Loss: 0.6501, Accuracy: 95.60%

Test set: Average loss: 0.6280, Accuracy: 9622/10000 (96.22%)

New best model saved with accuracy: 96.22%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.571340
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.636193
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.611649
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.572159
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.617735
Train Epoch: 4 - Loss: 0.6160, Accuracy: 96.75%

Test set: Average loss: 0.5995, Accuracy: 9713/10000 (97.13%)

New best model saved with accuracy: 97.13%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.563702
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.609625
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.607071
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.583993
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.583306
Train Epoch: 5 - Loss: 0.5967, Accuracy: 97.38%

Test set: Average loss: 0.5843, Accuracy: 9756/10000 (97.56%)

New best model saved with accuracy: 97.56%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.589446
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.565833
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.550661
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.617350
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.565746
Train Epoch: 6 - Loss: 0.5845, Accuracy: 97.77%

Test set: Average loss: 0.5802, Accuracy: 9767/10000 (97.67%)

New best model saved with accuracy: 97.67%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.604653
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.598115
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.532693
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.622201
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.575747
Train Epoch: 7 - Loss: 0.5762, Accuracy: 97.96%

Test set: Average loss: 0.5712, Accuracy: 9799/10000 (97.99%)

New best model saved with accuracy: 97.99%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.572475
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.621511
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.573672
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.567306
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.570178
Train Epoch: 8 - Loss: 0.5701, Accuracy: 98.14%

Test set: Average loss: 0.5668, Accuracy: 9805/10000 (98.05%)

New best model saved with accuracy: 98.05%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.548760
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.550522
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.547930
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.558513
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.557885
Train Epoch: 9 - Loss: 0.5654, Accuracy: 98.31%

Test set: Average loss: 0.5639, Accuracy: 9821/10000 (98.21%)

New best model saved with accuracy: 98.21%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.545110
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.579200
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.573267
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.584942
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.573862
Train Epoch: 10 - Loss: 0.5616, Accuracy: 98.43%

Test set: Average loss: 0.5603, Accuracy: 9828/10000 (98.28%)

New best model saved with accuracy: 98.28%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.571302
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.561138
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.564514
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.540889
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.525760
Train Epoch: 11 - Loss: 0.5583, Accuracy: 98.55%

Test set: Average loss: 0.5585, Accuracy: 9833/10000 (98.33%)

New best model saved with accuracy: 98.33%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.548199
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.559353
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.545401
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.537140
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.539306
Train Epoch: 12 - Loss: 0.5558, Accuracy: 98.64%

Test set: Average loss: 0.5557, Accuracy: 9841/10000 (98.41%)

New best model saved with accuracy: 98.41%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.555425
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.545194
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.539762
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.551092
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.562022
Train Epoch: 13 - Loss: 0.5536, Accuracy: 98.65%

Test set: Average loss: 0.5548, Accuracy: 9849/10000 (98.49%)

New best model saved with accuracy: 98.49%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.546637
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.558993
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.573492
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.556597
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.544523
Train Epoch: 14 - Loss: 0.5518, Accuracy: 98.74%

Test set: Average loss: 0.5536, Accuracy: 9848/10000 (98.48%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.546783
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.550991
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.534709
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.561483
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.569725
Train Epoch: 15 - Loss: 0.5504, Accuracy: 98.77%

Test set: Average loss: 0.5521, Accuracy: 9850/10000 (98.50%)

New best model saved with accuracy: 98.50%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.530595
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.552049
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.550928
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.533849
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.545605
Train Epoch: 16 - Loss: 0.5491, Accuracy: 98.79%

Test set: Average loss: 0.5517, Accuracy: 9853/10000 (98.53%)

New best model saved with accuracy: 98.53%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.559196
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.542549
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.525573
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.537402
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.528248
Train Epoch: 17 - Loss: 0.5481, Accuracy: 98.83%

Test set: Average loss: 0.5509, Accuracy: 9857/10000 (98.57%)

New best model saved with accuracy: 98.57%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.551996
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.559703
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.528970
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.541659
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.542943
Train Epoch: 18 - Loss: 0.5473, Accuracy: 98.85%

Test set: Average loss: 0.5500, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.539140
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.517818
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.529902
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.542049
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.533190
Train Epoch: 19 - Loss: 0.5465, Accuracy: 98.89%

Test set: Average loss: 0.5496, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.532209
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.559108
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.534611
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.527937
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.553566
Train Epoch: 20 - Loss: 0.5460, Accuracy: 98.90%

Test set: Average loss: 0.5497, Accuracy: 9853/10000 (98.53%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.543443
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.528371
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.537924
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.542166
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.533353
Train Epoch: 21 - Loss: 0.5456, Accuracy: 98.93%

Test set: Average loss: 0.5493, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.552400
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.563308
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.530826
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.539818
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.553198
Train Epoch: 22 - Loss: 0.5453, Accuracy: 98.92%

Test set: Average loss: 0.5491, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.556046
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.527076
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.569506
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.551678
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.536212
Train Epoch: 23 - Loss: 0.5451, Accuracy: 98.93%

Test set: Average loss: 0.5491, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.544751
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.544051
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.542007
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.554215
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.549592
Train Epoch: 24 - Loss: 0.5450, Accuracy: 98.94%

Test set: Average loss: 0.5490, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.545264
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.526094
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.539879
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.525840
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.534213
Train Epoch: 25 - Loss: 0.5449, Accuracy: 98.94%

Test set: Average loss: 0.5489, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.543222
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.558410
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.541866
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.520477
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.564342
Train Epoch: 26 - Loss: 0.5476, Accuracy: 98.85%

Test set: Average loss: 0.5497, Accuracy: 9851/10000 (98.51%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.573973
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.551973
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.530981
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.537916
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.549634
Train Epoch: 27 - Loss: 0.5455, Accuracy: 98.91%

Test set: Average loss: 0.5481, Accuracy: 9852/10000 (98.52%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.530547
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.579314
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.530914
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.549782
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.529708
Train Epoch: 28 - Loss: 0.5436, Accuracy: 98.94%

Test set: Average loss: 0.5464, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.547217
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.556677
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.544702
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.519465
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.545718
Train Epoch: 29 - Loss: 0.5418, Accuracy: 98.98%

Test set: Average loss: 0.5449, Accuracy: 9870/10000 (98.70%)

New best model saved with accuracy: 98.70%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.545643
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.556196
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.530100
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.539624
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.527337
Train Epoch: 30 - Loss: 0.5401, Accuracy: 99.06%

Test set: Average loss: 0.5465, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.537723
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.527890
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.544335
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.529471
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.551544
Train Epoch: 31 - Loss: 0.5386, Accuracy: 99.11%

Test set: Average loss: 0.5423, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.532480
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.527689
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.562020
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.548369
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.522840
Train Epoch: 32 - Loss: 0.5371, Accuracy: 99.15%

Test set: Average loss: 0.5416, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.545998
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.524701
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.560470
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.544644
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.519630
Train Epoch: 33 - Loss: 0.5360, Accuracy: 99.20%

Test set: Average loss: 0.5417, Accuracy: 9871/10000 (98.71%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.518856
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.518514
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.522109
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.530820
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.542851
Train Epoch: 34 - Loss: 0.5350, Accuracy: 99.21%

Test set: Average loss: 0.5404, Accuracy: 9883/10000 (98.83%)

New best model saved with accuracy: 98.83%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.519776
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.523782
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.527909
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.554965
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.539476
Train Epoch: 35 - Loss: 0.5339, Accuracy: 99.22%

Test set: Average loss: 0.5391, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.529742
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.529007
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.552900
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.557048
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.518724
Train Epoch: 36 - Loss: 0.5330, Accuracy: 99.27%

Test set: Average loss: 0.5390, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.518827
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.533452
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.538255
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.517346
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.529716
Train Epoch: 37 - Loss: 0.5323, Accuracy: 99.28%

Test set: Average loss: 0.5399, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.532938
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.536937
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.534181
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.531947
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.561261
Train Epoch: 38 - Loss: 0.5316, Accuracy: 99.31%

Test set: Average loss: 0.5386, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.520041
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.528458
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.532936
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.516470
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.525339
Train Epoch: 39 - Loss: 0.5309, Accuracy: 99.32%

Test set: Average loss: 0.5379, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.523690
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.530826
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.534643
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.546650
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.525164
Train Epoch: 40 - Loss: 0.5303, Accuracy: 99.36%

Test set: Average loss: 0.5380, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.525124
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.523057
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.523132
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.526690
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.524731
Train Epoch: 41 - Loss: 0.5299, Accuracy: 99.36%

Test set: Average loss: 0.5371, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.532714
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.551014
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.513827
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.543741
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.519853
Train Epoch: 42 - Loss: 0.5294, Accuracy: 99.40%

Test set: Average loss: 0.5372, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.519426
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.522040
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.549540
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.532854
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.529056
Train Epoch: 43 - Loss: 0.5291, Accuracy: 99.39%

Test set: Average loss: 0.5371, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.519254
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.543296
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.536131
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.530175
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.521719
Train Epoch: 44 - Loss: 0.5288, Accuracy: 99.39%

Test set: Average loss: 0.5368, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.539968
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.530045
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.523099
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.525301
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.536190
Train Epoch: 45 - Loss: 0.5285, Accuracy: 99.41%

Test set: Average loss: 0.5367, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.520826
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.533441
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.518603
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.521547
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.547808
Train Epoch: 46 - Loss: 0.5283, Accuracy: 99.42%

Test set: Average loss: 0.5366, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.534043
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.531994
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.529545
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.529759
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.527861
Train Epoch: 47 - Loss: 0.5282, Accuracy: 99.42%

Test set: Average loss: 0.5366, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.522048
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.530391
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.521681
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.526091
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.531606
Train Epoch: 48 - Loss: 0.5281, Accuracy: 99.44%

Test set: Average loss: 0.5365, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.523751
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.530058
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.521908
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.519376
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.513261
Train Epoch: 49 - Loss: 0.5280, Accuracy: 99.44%

Test set: Average loss: 0.5365, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.518966
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.538685
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.520001
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.526456
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.517639
Train Epoch: 50 - Loss: 0.5280, Accuracy: 99.44%

Test set: Average loss: 0.5365, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.523767
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.535689
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.524288
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.523186
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.540955
Train Epoch: 51 - Loss: 0.5303, Accuracy: 99.34%

Test set: Average loss: 0.5383, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.521581
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.531444
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.522487
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.538813
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.525548
Train Epoch: 52 - Loss: 0.5295, Accuracy: 99.36%

Test set: Average loss: 0.5386, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.518318
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.522966
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.522607
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.520782
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.537266
Train Epoch: 53 - Loss: 0.5286, Accuracy: 99.39%

Test set: Average loss: 0.5379, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.534371
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.523404
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.518326
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.514244
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.533928
Train Epoch: 54 - Loss: 0.5277, Accuracy: 99.43%

Test set: Average loss: 0.5386, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.523442
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.520127
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.542412
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.521204
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.524989
Train Epoch: 55 - Loss: 0.5270, Accuracy: 99.45%

Test set: Average loss: 0.5348, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.512748
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.536603
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.516535
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.514912
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.531537
Train Epoch: 56 - Loss: 0.5263, Accuracy: 99.45%

Test set: Average loss: 0.5351, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.528069
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.511721
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.543267
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.522000
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.520897
Train Epoch: 57 - Loss: 0.5257, Accuracy: 99.48%

Test set: Average loss: 0.5341, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.521747
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.528086
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.526870
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.527776
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.513320
Train Epoch: 58 - Loss: 0.5250, Accuracy: 99.50%

Test set: Average loss: 0.5340, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.510303
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.535474
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.521432
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.510696
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.509792
Train Epoch: 59 - Loss: 0.5245, Accuracy: 99.50%

Test set: Average loss: 0.5336, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.515767
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.539160
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.538041
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.518897
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.514528
Train Epoch: 60 - Loss: 0.5238, Accuracy: 99.55%

Test set: Average loss: 0.5337, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.518222
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.517005
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.516223
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.532875
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.513812
Train Epoch: 61 - Loss: 0.5235, Accuracy: 99.55%

Test set: Average loss: 0.5339, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.536754
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.525621
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.524163
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.519906
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.536301
Train Epoch: 62 - Loss: 0.5229, Accuracy: 99.58%

Test set: Average loss: 0.5336, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.545282
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.538448
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.530763
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.511520
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.543232
Train Epoch: 63 - Loss: 0.5225, Accuracy: 99.58%

Test set: Average loss: 0.5326, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.523693
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.517508
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.532423
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.516716
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.512043
Train Epoch: 64 - Loss: 0.5221, Accuracy: 99.60%

Test set: Average loss: 0.5327, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.520321
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.526710
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.521020
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.532956
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.538449
Train Epoch: 65 - Loss: 0.5218, Accuracy: 99.60%

Test set: Average loss: 0.5321, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.516748
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.543092
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.519236
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.520159
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.510095
Train Epoch: 66 - Loss: 0.5215, Accuracy: 99.60%

Test set: Average loss: 0.5321, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.517832
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.550625
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.551927
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.531664
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.524540
Train Epoch: 67 - Loss: 0.5212, Accuracy: 99.62%

Test set: Average loss: 0.5321, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.538418
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.528756
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.513446
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.521530
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.529081
Train Epoch: 68 - Loss: 0.5210, Accuracy: 99.63%

Test set: Average loss: 0.5318, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.511450
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.519118
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.534034
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.513100
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.510971
Train Epoch: 69 - Loss: 0.5208, Accuracy: 99.63%

Test set: Average loss: 0.5317, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.518560
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.528675
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.532966
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.520165
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.521511
Train Epoch: 70 - Loss: 0.5207, Accuracy: 99.64%

Test set: Average loss: 0.5316, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.513085
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.514308
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.519533
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.535843
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.525959
Train Epoch: 71 - Loss: 0.5205, Accuracy: 99.65%

Test set: Average loss: 0.5318, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.521872
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.520678
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.515984
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.530793
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.508821
Train Epoch: 72 - Loss: 0.5204, Accuracy: 99.65%

Test set: Average loss: 0.5317, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.514368
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.515201
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508542
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.534139
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.521739
Train Epoch: 73 - Loss: 0.5204, Accuracy: 99.66%

Test set: Average loss: 0.5316, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.536062
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.527330
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.518180
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.517184
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513954
Train Epoch: 74 - Loss: 0.5203, Accuracy: 99.66%

Test set: Average loss: 0.5316, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.536259
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.518880
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.519662
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.519861
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.524906
Train Epoch: 75 - Loss: 0.5203, Accuracy: 99.66%

Test set: Average loss: 0.5316, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.524174
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.521495
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.510610
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.512523
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.517706
Train Epoch: 76 - Loss: 0.5220, Accuracy: 99.60%

Test set: Average loss: 0.5322, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.511204
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.522266
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.521416
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.521201
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.526585
Train Epoch: 77 - Loss: 0.5216, Accuracy: 99.61%

Test set: Average loss: 0.5320, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.526054
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.518727
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.516235
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.523581
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.532833
Train Epoch: 78 - Loss: 0.5211, Accuracy: 99.61%

Test set: Average loss: 0.5329, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.517619
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.521810
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.511848
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.508099
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.524467
Train Epoch: 79 - Loss: 0.5205, Accuracy: 99.64%

Test set: Average loss: 0.5317, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.516629
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.510059
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.511284
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.519445
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.515131
Train Epoch: 80 - Loss: 0.5202, Accuracy: 99.65%

Test set: Average loss: 0.5312, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.514673
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.527512
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.538808
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.519049
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.508811
Train Epoch: 81 - Loss: 0.5197, Accuracy: 99.66%

Test set: Average loss: 0.5314, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.510468
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.518120
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.513536
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.511176
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.536526
Train Epoch: 82 - Loss: 0.5193, Accuracy: 99.65%

Test set: Average loss: 0.5314, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.518533
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.521146
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.517736
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.517729
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.526918
Train Epoch: 83 - Loss: 0.5189, Accuracy: 99.67%

Test set: Average loss: 0.5305, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.517856
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.534041
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.509808
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.514693
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.520771
Train Epoch: 84 - Loss: 0.5185, Accuracy: 99.68%

Test set: Average loss: 0.5305, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.518485
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.509991
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.512067
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.508817
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.511786
Train Epoch: 85 - Loss: 0.5181, Accuracy: 99.69%

Test set: Average loss: 0.5305, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.511985
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.522667
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.531015
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.518386
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.511032
Train Epoch: 86 - Loss: 0.5179, Accuracy: 99.70%

Test set: Average loss: 0.5301, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.513963
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.525962
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.510150
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.523734
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.512757
Train Epoch: 87 - Loss: 0.5175, Accuracy: 99.71%

Test set: Average loss: 0.5294, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.507100
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.525683
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.514045
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.525555
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.517683
Train Epoch: 88 - Loss: 0.5172, Accuracy: 99.72%

Test set: Average loss: 0.5291, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.533374
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.525088
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.515112
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.515459
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.517748
Train Epoch: 89 - Loss: 0.5170, Accuracy: 99.73%

Test set: Average loss: 0.5295, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.517998
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513579
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.518974
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.511303
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.512200
Train Epoch: 90 - Loss: 0.5167, Accuracy: 99.74%

Test set: Average loss: 0.5296, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.517526
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.525129
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.508385
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.526353
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.517633
Train Epoch: 91 - Loss: 0.5165, Accuracy: 99.75%

Test set: Average loss: 0.5294, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.517571
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.518627
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510892
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.518383
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.522044
Train Epoch: 92 - Loss: 0.5163, Accuracy: 99.74%

Test set: Average loss: 0.5292, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.513590
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.517732
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.539348
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.519698
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.519920
Train Epoch: 93 - Loss: 0.5162, Accuracy: 99.76%

Test set: Average loss: 0.5295, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.517973
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.516948
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.512463
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.525300
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.509724
Train Epoch: 94 - Loss: 0.5160, Accuracy: 99.75%

Test set: Average loss: 0.5289, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.513586
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.514690
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.521581
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.515819
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509488
Train Epoch: 95 - Loss: 0.5159, Accuracy: 99.77%

Test set: Average loss: 0.5290, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.512238
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.517686
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.515180
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.509360
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.519762
Train Epoch: 96 - Loss: 0.5158, Accuracy: 99.77%

Test set: Average loss: 0.5289, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.520107
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.519089
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.508894
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.515214
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.516830
Train Epoch: 97 - Loss: 0.5158, Accuracy: 99.77%

Test set: Average loss: 0.5289, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.513355
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.516170
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.511925
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.513234
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.513336
Train Epoch: 98 - Loss: 0.5157, Accuracy: 99.77%

Test set: Average loss: 0.5289, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.525730
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.509379
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.508300
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.513334
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.519019
Train Epoch: 99 - Loss: 0.5157, Accuracy: 99.78%

Test set: Average loss: 0.5289, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.517180
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.513314
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.507025
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.525996
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.518744
Train Epoch: 100 - Loss: 0.5156, Accuracy: 99.77%

Test set: Average loss: 0.5288, Accuracy: 9902/10000 (99.02%)


Training completed. Best Accuracy: 99.03%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.309219
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.481349
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.840578
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.822924
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.794232
Train Epoch: 1 - Loss: 1.1470, Accuracy: 77.38%

Test set: Average loss: 0.7628, Accuracy: 9166/10000 (91.66%)

New best model saved with accuracy: 91.66%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.805483
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.753998
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.789600
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.659060
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.683362
Train Epoch: 2 - Loss: 0.7226, Accuracy: 93.00%

Test set: Average loss: 0.6672, Accuracy: 9502/10000 (95.02%)

New best model saved with accuracy: 95.02%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.605999
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.662045
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.719795
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.662022
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.633712
Train Epoch: 3 - Loss: 0.6516, Accuracy: 95.51%

Test set: Average loss: 0.6246, Accuracy: 9646/10000 (96.46%)

New best model saved with accuracy: 96.46%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.596218
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.574622
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.645488
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.613055
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.592518
Train Epoch: 4 - Loss: 0.6163, Accuracy: 96.64%

Test set: Average loss: 0.5992, Accuracy: 9711/10000 (97.11%)

New best model saved with accuracy: 97.11%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.649118
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.582246
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.622114
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.596524
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.582536
Train Epoch: 5 - Loss: 0.5967, Accuracy: 97.30%

Test set: Average loss: 0.5888, Accuracy: 9741/10000 (97.41%)

New best model saved with accuracy: 97.41%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.610511
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.566144
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.568559
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.604015
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.582961
Train Epoch: 6 - Loss: 0.5849, Accuracy: 97.68%

Test set: Average loss: 0.5829, Accuracy: 9764/10000 (97.64%)

New best model saved with accuracy: 97.64%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.570506
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.594539
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.621701
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.597105
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.578748
Train Epoch: 7 - Loss: 0.5763, Accuracy: 97.94%

Test set: Average loss: 0.5715, Accuracy: 9783/10000 (97.83%)

New best model saved with accuracy: 97.83%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.558851
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.559690
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.566268
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.574116
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.569996
Train Epoch: 8 - Loss: 0.5697, Accuracy: 98.15%

Test set: Average loss: 0.5658, Accuracy: 9817/10000 (98.17%)

New best model saved with accuracy: 98.17%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.545275
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.539411
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.561916
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.578423
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.587142
Train Epoch: 9 - Loss: 0.5648, Accuracy: 98.29%

Test set: Average loss: 0.5634, Accuracy: 9827/10000 (98.27%)

New best model saved with accuracy: 98.27%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.536389
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.537509
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.564791
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.600024
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.559097
Train Epoch: 10 - Loss: 0.5608, Accuracy: 98.43%

Test set: Average loss: 0.5589, Accuracy: 9824/10000 (98.24%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.545036
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.595571
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.590140
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.550909
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.549557
Train Epoch: 11 - Loss: 0.5576, Accuracy: 98.51%

Test set: Average loss: 0.5571, Accuracy: 9833/10000 (98.33%)

New best model saved with accuracy: 98.33%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.542889
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.558382
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.532029
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.534598
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.600411
Train Epoch: 12 - Loss: 0.5550, Accuracy: 98.59%

Test set: Average loss: 0.5555, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.534807
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.545934
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.530671
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.545759
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.523813
Train Epoch: 13 - Loss: 0.5527, Accuracy: 98.67%

Test set: Average loss: 0.5530, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.544934
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.538376
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.534009
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.583803
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.557901
Train Epoch: 14 - Loss: 0.5510, Accuracy: 98.71%

Test set: Average loss: 0.5528, Accuracy: 9841/10000 (98.41%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.556220
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.520850
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.528347
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.562856
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.536297
Train Epoch: 15 - Loss: 0.5494, Accuracy: 98.77%

Test set: Average loss: 0.5509, Accuracy: 9846/10000 (98.46%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.530482
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.545221
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.547842
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.527737
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.539780
Train Epoch: 16 - Loss: 0.5481, Accuracy: 98.83%

Test set: Average loss: 0.5516, Accuracy: 9847/10000 (98.47%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.523391
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.534468
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.539316
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.571028
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.557653
Train Epoch: 17 - Loss: 0.5470, Accuracy: 98.84%

Test set: Average loss: 0.5495, Accuracy: 9853/10000 (98.53%)

New best model saved with accuracy: 98.53%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.539822
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.545139
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.565768
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.545519
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.550867
Train Epoch: 18 - Loss: 0.5462, Accuracy: 98.86%

Test set: Average loss: 0.5490, Accuracy: 9858/10000 (98.58%)

New best model saved with accuracy: 98.58%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.538790
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.551255
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.554181
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.540263
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.544327
Train Epoch: 19 - Loss: 0.5455, Accuracy: 98.88%

Test set: Average loss: 0.5483, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.545177
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.534582
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.560991
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.535376
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.537832
Train Epoch: 20 - Loss: 0.5450, Accuracy: 98.91%

Test set: Average loss: 0.5481, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.557808
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.539096
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.551618
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.546844
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.548609
Train Epoch: 21 - Loss: 0.5445, Accuracy: 98.92%

Test set: Average loss: 0.5480, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.528289
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.533046
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.594502
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.524896
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.551116
Train Epoch: 22 - Loss: 0.5443, Accuracy: 98.92%

Test set: Average loss: 0.5477, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.544463
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.543247
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.544693
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.556824
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.542312
Train Epoch: 23 - Loss: 0.5441, Accuracy: 98.94%

Test set: Average loss: 0.5476, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.547719
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.554222
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.583397
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.524062
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.531290
Train Epoch: 24 - Loss: 0.5439, Accuracy: 98.95%

Test set: Average loss: 0.5476, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.584883
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.544419
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.542495
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.561545
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.537823
Train Epoch: 25 - Loss: 0.5439, Accuracy: 98.96%

Test set: Average loss: 0.5476, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.537151
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.535569
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.527644
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.537226
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.560091
Train Epoch: 26 - Loss: 0.5462, Accuracy: 98.85%

Test set: Average loss: 0.5491, Accuracy: 9859/10000 (98.59%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.527502
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.563017
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.534024
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.528354
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.534850
Train Epoch: 27 - Loss: 0.5441, Accuracy: 98.91%

Test set: Average loss: 0.5458, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.532669
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.534042
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.529526
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.530614
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.523703
Train Epoch: 28 - Loss: 0.5418, Accuracy: 98.97%

Test set: Average loss: 0.5450, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.527263
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.533098
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.535727
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.532078
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.536587
Train Epoch: 29 - Loss: 0.5401, Accuracy: 99.06%

Test set: Average loss: 0.5442, Accuracy: 9868/10000 (98.68%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.538783
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.528323
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.525389
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.533565
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.525916
Train Epoch: 30 - Loss: 0.5386, Accuracy: 99.08%

Test set: Average loss: 0.5423, Accuracy: 9873/10000 (98.73%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.531504
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.528953
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.540873
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.529182
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.529036
Train Epoch: 31 - Loss: 0.5371, Accuracy: 99.15%

Test set: Average loss: 0.5439, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.553355
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.529775
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.532249
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.536292
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.532064
Train Epoch: 32 - Loss: 0.5357, Accuracy: 99.17%

Test set: Average loss: 0.5422, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.537276
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.526210
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.528852
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.526562
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.526660
Train Epoch: 33 - Loss: 0.5345, Accuracy: 99.21%

Test set: Average loss: 0.5395, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.527610
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.526667
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.570802
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.523887
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.514511
Train Epoch: 34 - Loss: 0.5334, Accuracy: 99.23%

Test set: Average loss: 0.5385, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.540755
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.527112
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.522073
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.525829
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.567436
Train Epoch: 35 - Loss: 0.5325, Accuracy: 99.27%

Test set: Average loss: 0.5395, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.515688
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.525886
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.534577
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.520130
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.516413
Train Epoch: 36 - Loss: 0.5315, Accuracy: 99.26%

Test set: Average loss: 0.5387, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.525245
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.528175
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.517317
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.512542
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.536748
Train Epoch: 37 - Loss: 0.5308, Accuracy: 99.30%

Test set: Average loss: 0.5373, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.516586
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.538937
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.533705
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.512670
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.530164
Train Epoch: 38 - Loss: 0.5301, Accuracy: 99.32%

Test set: Average loss: 0.5362, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.514752
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.534913
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.516465
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.533332
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.520436
Train Epoch: 39 - Loss: 0.5295, Accuracy: 99.35%

Test set: Average loss: 0.5366, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.512761
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.525282
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.519671
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.511657
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.550215
Train Epoch: 40 - Loss: 0.5289, Accuracy: 99.36%

Test set: Average loss: 0.5362, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.519008
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.517141
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.525102
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.532767
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.532424
Train Epoch: 41 - Loss: 0.5285, Accuracy: 99.39%

Test set: Average loss: 0.5360, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.535302
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.514490
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.524236
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.517378
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.515808
Train Epoch: 42 - Loss: 0.5281, Accuracy: 99.40%

Test set: Average loss: 0.5356, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.517066
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.519765
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.515287
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.532162
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.532819
Train Epoch: 43 - Loss: 0.5277, Accuracy: 99.42%

Test set: Average loss: 0.5355, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.515391
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.514412
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.519429
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.529872
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.526113
Train Epoch: 44 - Loss: 0.5275, Accuracy: 99.42%

Test set: Average loss: 0.5351, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.524896
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.511938
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.549798
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.518637
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.512485
Train Epoch: 45 - Loss: 0.5272, Accuracy: 99.44%

Test set: Average loss: 0.5350, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.527865
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.541182
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.525169
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.526633
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.519569
Train Epoch: 46 - Loss: 0.5271, Accuracy: 99.45%

Test set: Average loss: 0.5349, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.514741
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.516083
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.536426
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.541447
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.525280
Train Epoch: 47 - Loss: 0.5269, Accuracy: 99.46%

Test set: Average loss: 0.5349, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.519460
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.527385
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.533110
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.526644
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.535193
Train Epoch: 48 - Loss: 0.5268, Accuracy: 99.46%

Test set: Average loss: 0.5348, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.514815
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.526854
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.517848
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.559743
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.515417
Train Epoch: 49 - Loss: 0.5268, Accuracy: 99.45%

Test set: Average loss: 0.5348, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.512524
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.557312
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.525399
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.540856
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.513115
Train Epoch: 50 - Loss: 0.5267, Accuracy: 99.45%

Test set: Average loss: 0.5348, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.553731
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.525843
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.521092
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.548601
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.517099
Train Epoch: 51 - Loss: 0.5287, Accuracy: 99.39%

Test set: Average loss: 0.5359, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.527704
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.525505
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.543001
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.522176
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.513287
Train Epoch: 52 - Loss: 0.5279, Accuracy: 99.41%

Test set: Average loss: 0.5349, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.514216
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.516869
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.543631
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.515133
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.536399
Train Epoch: 53 - Loss: 0.5271, Accuracy: 99.42%

Test set: Average loss: 0.5352, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.527078
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.544633
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.529780
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.512245
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.515989
Train Epoch: 54 - Loss: 0.5264, Accuracy: 99.45%

Test set: Average loss: 0.5343, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.521424
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.545271
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.520165
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.512944
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.520650
Train Epoch: 55 - Loss: 0.5256, Accuracy: 99.48%

Test set: Average loss: 0.5338, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.533144
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.513518
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514787
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.529965
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.524080
Train Epoch: 56 - Loss: 0.5251, Accuracy: 99.48%

Test set: Average loss: 0.5326, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.515026
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.517161
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.519582
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.550494
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.528688
Train Epoch: 57 - Loss: 0.5243, Accuracy: 99.50%

Test set: Average loss: 0.5323, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.516722
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.513601
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.523206
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.512429
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.518345
Train Epoch: 58 - Loss: 0.5237, Accuracy: 99.53%

Test set: Average loss: 0.5320, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.522997
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.534328
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.520275
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.520035
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.511276
Train Epoch: 59 - Loss: 0.5232, Accuracy: 99.55%

Test set: Average loss: 0.5321, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.530578
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.526349
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.524407
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.519588
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.531422
Train Epoch: 60 - Loss: 0.5227, Accuracy: 99.57%

Test set: Average loss: 0.5311, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.559705
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.524423
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.521996
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.536176
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.516170
Train Epoch: 61 - Loss: 0.5222, Accuracy: 99.59%

Test set: Average loss: 0.5306, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.516787
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.523939
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.515693
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.522813
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.530079
Train Epoch: 62 - Loss: 0.5218, Accuracy: 99.60%

Test set: Average loss: 0.5308, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.525829
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.517699
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.529390
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.510062
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.553339
Train Epoch: 63 - Loss: 0.5213, Accuracy: 99.60%

Test set: Average loss: 0.5309, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.528018
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.528658
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.514026
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.531494
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.508493
Train Epoch: 64 - Loss: 0.5210, Accuracy: 99.62%

Test set: Average loss: 0.5305, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.512257
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.531046
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.521976
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.527676
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.514902
Train Epoch: 65 - Loss: 0.5207, Accuracy: 99.61%

Test set: Average loss: 0.5302, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.508207
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.513539
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.527453
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.535729
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.522513
Train Epoch: 66 - Loss: 0.5204, Accuracy: 99.63%

Test set: Average loss: 0.5303, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.518931
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.521722
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.531413
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.509928
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.512375
Train Epoch: 67 - Loss: 0.5202, Accuracy: 99.64%

Test set: Average loss: 0.5302, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.513027
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.523976
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.533726
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.518381
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.517578
Train Epoch: 68 - Loss: 0.5199, Accuracy: 99.63%

Test set: Average loss: 0.5301, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.515494
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.516950
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.511915
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.514074
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.511750
Train Epoch: 69 - Loss: 0.5198, Accuracy: 99.64%

Test set: Average loss: 0.5299, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.520114
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.513961
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.515733
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.515237
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.518857
Train Epoch: 70 - Loss: 0.5196, Accuracy: 99.65%

Test set: Average loss: 0.5300, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.518041
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.513315
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.519626
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.518545
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.524847
Train Epoch: 71 - Loss: 0.5195, Accuracy: 99.66%

Test set: Average loss: 0.5298, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.512413
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509982
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.523650
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.539593
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.514146
Train Epoch: 72 - Loss: 0.5194, Accuracy: 99.66%

Test set: Average loss: 0.5298, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.509225
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.526833
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.519738
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.515662
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.512880
Train Epoch: 73 - Loss: 0.5193, Accuracy: 99.66%

Test set: Average loss: 0.5298, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.520451
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.527095
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.510470
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.514462
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.524228
Train Epoch: 74 - Loss: 0.5193, Accuracy: 99.66%

Test set: Average loss: 0.5298, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.517059
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.511163
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.510004
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.513980
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.520276
Train Epoch: 75 - Loss: 0.5193, Accuracy: 99.66%

Test set: Average loss: 0.5298, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.515479
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.527103
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.515939
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.535573
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.511600
Train Epoch: 76 - Loss: 0.5209, Accuracy: 99.62%

Test set: Average loss: 0.5307, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.512719
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.514586
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.512036
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.522604
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.521443
Train Epoch: 77 - Loss: 0.5204, Accuracy: 99.63%

Test set: Average loss: 0.5313, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.533018
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.517310
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.521123
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.514702
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.510443
Train Epoch: 78 - Loss: 0.5199, Accuracy: 99.63%

Test set: Average loss: 0.5297, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.508471
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.512997
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.509484
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.519465
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.510670
Train Epoch: 79 - Loss: 0.5195, Accuracy: 99.63%

Test set: Average loss: 0.5295, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.513448
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.509991
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.525342
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.532978
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.543971
Train Epoch: 80 - Loss: 0.5191, Accuracy: 99.66%

Test set: Average loss: 0.5289, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.529205
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.527829
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.529161
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.519555
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.510441
Train Epoch: 81 - Loss: 0.5187, Accuracy: 99.66%

Test set: Average loss: 0.5288, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.514347
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.517073
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.529322
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.512598
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.513145
Train Epoch: 82 - Loss: 0.5183, Accuracy: 99.69%

Test set: Average loss: 0.5311, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.519046
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.532441
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.514765
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.531476
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.517479
Train Epoch: 83 - Loss: 0.5179, Accuracy: 99.68%

Test set: Average loss: 0.5295, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.514592
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.517083
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.508232
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.511648
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.511090
Train Epoch: 84 - Loss: 0.5175, Accuracy: 99.70%

Test set: Average loss: 0.5285, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.510284
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.512701
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.511438
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.515177
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.516420
Train Epoch: 85 - Loss: 0.5173, Accuracy: 99.70%

Test set: Average loss: 0.5281, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.515390
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.510428
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.506459
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.517201
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.509803
Train Epoch: 86 - Loss: 0.5169, Accuracy: 99.71%

Test set: Average loss: 0.5285, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.513329
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.517201
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.516569
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.512181
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.537159
Train Epoch: 87 - Loss: 0.5166, Accuracy: 99.72%

Test set: Average loss: 0.5282, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.513317
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.509513
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.520006
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.516030
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.506625
Train Epoch: 88 - Loss: 0.5163, Accuracy: 99.72%

Test set: Average loss: 0.5280, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.511236
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.520987
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.509035
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.517330
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.510573
Train Epoch: 89 - Loss: 0.5161, Accuracy: 99.74%

Test set: Average loss: 0.5275, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.514788
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.514850
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.526664
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.512579
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.509812
Train Epoch: 90 - Loss: 0.5159, Accuracy: 99.75%

Test set: Average loss: 0.5272, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.511792
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.509615
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.510951
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.507206
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.524259
Train Epoch: 91 - Loss: 0.5157, Accuracy: 99.75%

Test set: Average loss: 0.5273, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.527277
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.515157
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.525305
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507251
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.511197
Train Epoch: 92 - Loss: 0.5155, Accuracy: 99.75%

Test set: Average loss: 0.5275, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.510097
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.512512
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.512129
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.517163
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.513323
Train Epoch: 93 - Loss: 0.5154, Accuracy: 99.75%

Test set: Average loss: 0.5273, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.516279
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.519711
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.517957
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.530293
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.511268
Train Epoch: 94 - Loss: 0.5152, Accuracy: 99.76%

Test set: Average loss: 0.5273, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.533554
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.509620
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.508657
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.507310
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509701
Train Epoch: 95 - Loss: 0.5151, Accuracy: 99.77%

Test set: Average loss: 0.5274, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.515517
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.511966
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.520598
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.510223
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.507633
Train Epoch: 96 - Loss: 0.5150, Accuracy: 99.77%

Test set: Average loss: 0.5271, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.506972
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.510637
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.524023
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.515926
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.519479
Train Epoch: 97 - Loss: 0.5150, Accuracy: 99.77%

Test set: Average loss: 0.5271, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.508857
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.511996
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.515879
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.513496
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.523806
Train Epoch: 98 - Loss: 0.5149, Accuracy: 99.77%

Test set: Average loss: 0.5271, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.505958
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.524291
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.518710
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.509291
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.507583
Train Epoch: 99 - Loss: 0.5149, Accuracy: 99.77%

Test set: Average loss: 0.5271, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.512809
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.508843
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508082
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.510367
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.511277
Train Epoch: 100 - Loss: 0.5149, Accuracy: 99.77%

Test set: Average loss: 0.5271, Accuracy: 9906/10000 (99.06%)


Training completed. Best Accuracy: 99.09%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.299646
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.263508
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.855712
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.863006
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.761204
Train Epoch: 1 - Loss: 1.1165, Accuracy: 78.92%

Test set: Average loss: 0.7550, Accuracy: 9230/10000 (92.30%)

New best model saved with accuracy: 92.30%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.740102
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.730342
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.745643
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.730812
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.657111
Train Epoch: 2 - Loss: 0.7149, Accuracy: 93.43%

Test set: Average loss: 0.6658, Accuracy: 9506/10000 (95.06%)

New best model saved with accuracy: 95.06%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.721825
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.643718
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.670456
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.614429
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.615436
Train Epoch: 3 - Loss: 0.6485, Accuracy: 95.67%

Test set: Average loss: 0.6222, Accuracy: 9669/10000 (96.69%)

New best model saved with accuracy: 96.69%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.565749
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.608438
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.616569
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.574210
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.614542
Train Epoch: 4 - Loss: 0.6157, Accuracy: 96.79%

Test set: Average loss: 0.6013, Accuracy: 9717/10000 (97.17%)

New best model saved with accuracy: 97.17%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.642671
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.608405
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.621485
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.577401
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.608531
Train Epoch: 5 - Loss: 0.5964, Accuracy: 97.41%

Test set: Average loss: 0.5855, Accuracy: 9766/10000 (97.66%)

New best model saved with accuracy: 97.66%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.573952
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.593161
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.588836
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.556731
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.599334
Train Epoch: 6 - Loss: 0.5844, Accuracy: 97.77%

Test set: Average loss: 0.5795, Accuracy: 9782/10000 (97.82%)

New best model saved with accuracy: 97.82%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.651339
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.589387
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.560601
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.583948
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.555550
Train Epoch: 7 - Loss: 0.5762, Accuracy: 97.98%

Test set: Average loss: 0.5725, Accuracy: 9802/10000 (98.02%)

New best model saved with accuracy: 98.02%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.574067
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.576701
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.577532
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.573234
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.567982
Train Epoch: 8 - Loss: 0.5699, Accuracy: 98.22%

Test set: Average loss: 0.5666, Accuracy: 9826/10000 (98.26%)

New best model saved with accuracy: 98.26%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.570874
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.568138
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.565860
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.585981
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.561344
Train Epoch: 9 - Loss: 0.5650, Accuracy: 98.33%

Test set: Average loss: 0.5620, Accuracy: 9828/10000 (98.28%)

New best model saved with accuracy: 98.28%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.544199
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.538524
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.566631
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.529764
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.542848
Train Epoch: 10 - Loss: 0.5611, Accuracy: 98.51%

Test set: Average loss: 0.5593, Accuracy: 9830/10000 (98.30%)

New best model saved with accuracy: 98.30%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.536995
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.570136
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.586579
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.544417
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.547090
Train Epoch: 11 - Loss: 0.5581, Accuracy: 98.57%

Test set: Average loss: 0.5587, Accuracy: 9822/10000 (98.22%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.537533
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.582200
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.571338
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.548178
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.529759
Train Epoch: 12 - Loss: 0.5554, Accuracy: 98.65%

Test set: Average loss: 0.5556, Accuracy: 9837/10000 (98.37%)

New best model saved with accuracy: 98.37%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.549254
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.563592
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.538955
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.549475
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.539729
Train Epoch: 13 - Loss: 0.5532, Accuracy: 98.71%

Test set: Average loss: 0.5548, Accuracy: 9842/10000 (98.42%)

New best model saved with accuracy: 98.42%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.522620
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.541856
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.532889
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.557254
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.587759
Train Epoch: 14 - Loss: 0.5514, Accuracy: 98.77%

Test set: Average loss: 0.5528, Accuracy: 9844/10000 (98.44%)

New best model saved with accuracy: 98.44%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.534900
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.526397
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.556748
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.535542
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.550207
Train Epoch: 15 - Loss: 0.5500, Accuracy: 98.82%

Test set: Average loss: 0.5523, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.540671
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.545220
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.541048
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.531261
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.531422
Train Epoch: 16 - Loss: 0.5488, Accuracy: 98.84%

Test set: Average loss: 0.5509, Accuracy: 9854/10000 (98.54%)

New best model saved with accuracy: 98.54%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.548408
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.559838
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.520476
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.535830
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.542206
Train Epoch: 17 - Loss: 0.5477, Accuracy: 98.85%

Test set: Average loss: 0.5503, Accuracy: 9852/10000 (98.52%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.564675
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.529689
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.550026
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.535427
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.536347
Train Epoch: 18 - Loss: 0.5468, Accuracy: 98.91%

Test set: Average loss: 0.5495, Accuracy: 9848/10000 (98.48%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.559180
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.581555
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.542842
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.545055
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.554212
Train Epoch: 19 - Loss: 0.5462, Accuracy: 98.94%

Test set: Average loss: 0.5490, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.544990
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.532975
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.586897
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.547553
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.560390
Train Epoch: 20 - Loss: 0.5457, Accuracy: 98.94%

Test set: Average loss: 0.5488, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.545375
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.528388
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.552738
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.584116
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.546483
Train Epoch: 21 - Loss: 0.5453, Accuracy: 98.94%

Test set: Average loss: 0.5486, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.532966
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.527284
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.532346
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.547985
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.537797
Train Epoch: 22 - Loss: 0.5450, Accuracy: 98.97%

Test set: Average loss: 0.5484, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.531515
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.532153
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.539761
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.540473
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.524002
Train Epoch: 23 - Loss: 0.5448, Accuracy: 98.97%

Test set: Average loss: 0.5483, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.565467
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.530350
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.531676
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.553466
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.587417
Train Epoch: 24 - Loss: 0.5447, Accuracy: 98.97%

Test set: Average loss: 0.5482, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.525890
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.541387
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.542768
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.551299
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.530020
Train Epoch: 25 - Loss: 0.5446, Accuracy: 98.98%

Test set: Average loss: 0.5482, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.541514
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.562318
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.549516
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.530896
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.549781
Train Epoch: 26 - Loss: 0.5470, Accuracy: 98.87%

Test set: Average loss: 0.5483, Accuracy: 9855/10000 (98.55%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.547795
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.548391
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.559075
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.542260
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.525026
Train Epoch: 27 - Loss: 0.5447, Accuracy: 98.98%

Test set: Average loss: 0.5464, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.530302
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.545576
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.555656
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.555772
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.544959
Train Epoch: 28 - Loss: 0.5426, Accuracy: 98.98%

Test set: Average loss: 0.5453, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.528875
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.539383
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.531823
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.535113
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.515056
Train Epoch: 29 - Loss: 0.5407, Accuracy: 99.06%

Test set: Average loss: 0.5441, Accuracy: 9867/10000 (98.67%)

New best model saved with accuracy: 98.67%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.528031
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.541345
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.554031
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.543009
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.528248
Train Epoch: 30 - Loss: 0.5393, Accuracy: 99.12%

Test set: Average loss: 0.5436, Accuracy: 9870/10000 (98.70%)

New best model saved with accuracy: 98.70%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.551327
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.530598
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.530187
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.526360
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.536677
Train Epoch: 31 - Loss: 0.5378, Accuracy: 99.13%

Test set: Average loss: 0.5419, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.551517
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.542191
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.526542
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.526092
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.527620
Train Epoch: 32 - Loss: 0.5365, Accuracy: 99.16%

Test set: Average loss: 0.5408, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.520907
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.540726
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.524351
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.540849
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.533479
Train Epoch: 33 - Loss: 0.5353, Accuracy: 99.19%

Test set: Average loss: 0.5410, Accuracy: 9883/10000 (98.83%)

New best model saved with accuracy: 98.83%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.532692
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.531749
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.546030
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.559554
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.528150
Train Epoch: 34 - Loss: 0.5342, Accuracy: 99.24%

Test set: Average loss: 0.5384, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.512270
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.541667
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.585517
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.532579
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.554697
Train Epoch: 35 - Loss: 0.5332, Accuracy: 99.26%

Test set: Average loss: 0.5378, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.519130
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.526959
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.550305
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.516446
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.530568
Train Epoch: 36 - Loss: 0.5324, Accuracy: 99.28%

Test set: Average loss: 0.5379, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.529520
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.553369
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.529567
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.519978
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.521209
Train Epoch: 37 - Loss: 0.5316, Accuracy: 99.30%

Test set: Average loss: 0.5376, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.539313
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.552607
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.529869
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.539172
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.531575
Train Epoch: 38 - Loss: 0.5308, Accuracy: 99.34%

Test set: Average loss: 0.5374, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.531963
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.529637
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.524195
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.543538
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.521973
Train Epoch: 39 - Loss: 0.5302, Accuracy: 99.35%

Test set: Average loss: 0.5365, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.565224
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.519536
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.520995
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.550107
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.539595
Train Epoch: 40 - Loss: 0.5297, Accuracy: 99.36%

Test set: Average loss: 0.5360, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.536674
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.539194
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.523410
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.528912
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.530556
Train Epoch: 41 - Loss: 0.5292, Accuracy: 99.39%

Test set: Average loss: 0.5358, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.515338
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.530748
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.542881
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.533525
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.534273
Train Epoch: 42 - Loss: 0.5288, Accuracy: 99.39%

Test set: Average loss: 0.5357, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.531496
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.541072
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.520356
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.522400
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.524942
Train Epoch: 43 - Loss: 0.5285, Accuracy: 99.42%

Test set: Average loss: 0.5354, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.518320
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.523512
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.524225
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.538891
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.527714
Train Epoch: 44 - Loss: 0.5282, Accuracy: 99.42%

Test set: Average loss: 0.5351, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.521112
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.510934
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.526135
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.524388
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.536339
Train Epoch: 45 - Loss: 0.5279, Accuracy: 99.42%

Test set: Average loss: 0.5350, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.520771
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.531668
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.530674
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.521715
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.516256
Train Epoch: 46 - Loss: 0.5278, Accuracy: 99.43%

Test set: Average loss: 0.5350, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.530554
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.549257
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.510033
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.556251
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.553329
Train Epoch: 47 - Loss: 0.5276, Accuracy: 99.45%

Test set: Average loss: 0.5349, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.523143
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.520976
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.523116
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.539796
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.523388
Train Epoch: 48 - Loss: 0.5275, Accuracy: 99.44%

Test set: Average loss: 0.5348, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.518435
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.539710
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.523398
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.533822
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.541600
Train Epoch: 49 - Loss: 0.5274, Accuracy: 99.45%

Test set: Average loss: 0.5348, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.521796
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.536452
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.521449
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.526747
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.543526
Train Epoch: 50 - Loss: 0.5274, Accuracy: 99.45%

Test set: Average loss: 0.5348, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.518015
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.517899
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.536320
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.518979
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.524572
Train Epoch: 51 - Loss: 0.5294, Accuracy: 99.39%

Test set: Average loss: 0.5366, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.549576
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.537513
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.536659
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.528882
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.513639
Train Epoch: 52 - Loss: 0.5284, Accuracy: 99.41%

Test set: Average loss: 0.5357, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.521558
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.537157
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.539137
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.523588
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.520786
Train Epoch: 53 - Loss: 0.5278, Accuracy: 99.42%

Test set: Average loss: 0.5337, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.545136
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.521178
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.534419
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.534070
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.525151
Train Epoch: 54 - Loss: 0.5271, Accuracy: 99.42%

Test set: Average loss: 0.5343, Accuracy: 9888/10000 (98.88%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.519155
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.515492
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.536515
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.515019
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.530883
Train Epoch: 55 - Loss: 0.5262, Accuracy: 99.46%

Test set: Average loss: 0.5336, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.512715
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.531938
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514875
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.532968
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.527979
Train Epoch: 56 - Loss: 0.5256, Accuracy: 99.47%

Test set: Average loss: 0.5333, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.514383
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.528696
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.529015
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.532722
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.541444
Train Epoch: 57 - Loss: 0.5249, Accuracy: 99.49%

Test set: Average loss: 0.5331, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.517351
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.539167
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.540178
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.526295
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.544347
Train Epoch: 58 - Loss: 0.5245, Accuracy: 99.48%

Test set: Average loss: 0.5320, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.525663
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.510241
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.536106
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.527717
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.516516
Train Epoch: 59 - Loss: 0.5239, Accuracy: 99.51%

Test set: Average loss: 0.5321, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.529327
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.535315
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.515565
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.513721
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.522636
Train Epoch: 60 - Loss: 0.5233, Accuracy: 99.53%

Test set: Average loss: 0.5312, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.519914
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.517000
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.512759
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.516456
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.513766
Train Epoch: 61 - Loss: 0.5229, Accuracy: 99.52%

Test set: Average loss: 0.5310, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.527697
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.517589
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.533535
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.538793
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.525811
Train Epoch: 62 - Loss: 0.5223, Accuracy: 99.56%

Test set: Average loss: 0.5320, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.524875
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.532231
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.514412
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.513088
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.524532
Train Epoch: 63 - Loss: 0.5220, Accuracy: 99.56%

Test set: Average loss: 0.5305, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.565160
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.516756
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.516922
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.522966
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.513948
Train Epoch: 64 - Loss: 0.5216, Accuracy: 99.59%

Test set: Average loss: 0.5304, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.510898
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.515765
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.561096
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.533256
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.515414
Train Epoch: 65 - Loss: 0.5213, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.512883
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.509486
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.518969
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.523328
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.514211
Train Epoch: 66 - Loss: 0.5210, Accuracy: 99.61%

Test set: Average loss: 0.5300, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.523380
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.510732
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.518675
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.530784
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.517856
Train Epoch: 67 - Loss: 0.5208, Accuracy: 99.59%

Test set: Average loss: 0.5298, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.511233
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.513495
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.528204
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.509635
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.512656
Train Epoch: 68 - Loss: 0.5206, Accuracy: 99.61%

Test set: Average loss: 0.5298, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.508612
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.527222
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.512737
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.539824
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.518761
Train Epoch: 69 - Loss: 0.5204, Accuracy: 99.62%

Test set: Average loss: 0.5298, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.522222
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.513951
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.525901
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.513923
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.523143
Train Epoch: 70 - Loss: 0.5203, Accuracy: 99.63%

Test set: Average loss: 0.5295, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.512580
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.514626
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.516749
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.511057
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.514547
Train Epoch: 71 - Loss: 0.5201, Accuracy: 99.63%

Test set: Average loss: 0.5296, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.519489
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.511636
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.517612
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.514242
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.515903
Train Epoch: 72 - Loss: 0.5200, Accuracy: 99.64%

Test set: Average loss: 0.5295, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.519591
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.511383
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.523224
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.513283
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.510277
Train Epoch: 73 - Loss: 0.5200, Accuracy: 99.64%

Test set: Average loss: 0.5295, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.513054
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.524749
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.512486
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.513470
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513105
Train Epoch: 74 - Loss: 0.5199, Accuracy: 99.64%

Test set: Average loss: 0.5295, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.563776
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.538120
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.516996
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.536774
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.521074
Train Epoch: 75 - Loss: 0.5199, Accuracy: 99.64%

Test set: Average loss: 0.5295, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.514919
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.510158
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.529086
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.521478
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.520341
Train Epoch: 76 - Loss: 0.5214, Accuracy: 99.59%

Test set: Average loss: 0.5301, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.514175
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.521880
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.519653
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.525202
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.511946
Train Epoch: 77 - Loss: 0.5210, Accuracy: 99.57%

Test set: Average loss: 0.5297, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.517908
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.519548
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.525203
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.517798
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.526107
Train Epoch: 78 - Loss: 0.5205, Accuracy: 99.60%

Test set: Average loss: 0.5299, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.529173
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.522098
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.518541
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.509181
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.517646
Train Epoch: 79 - Loss: 0.5201, Accuracy: 99.61%

Test set: Average loss: 0.5299, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.529384
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.512855
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.517359
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.527349
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.524261
Train Epoch: 80 - Loss: 0.5198, Accuracy: 99.63%

Test set: Average loss: 0.5291, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.511614
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.510487
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.512124
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.511954
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.515431
Train Epoch: 81 - Loss: 0.5193, Accuracy: 99.65%

Test set: Average loss: 0.5288, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.522367
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.536103
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.509839
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.533869
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.514720
Train Epoch: 82 - Loss: 0.5189, Accuracy: 99.64%

Test set: Average loss: 0.5288, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.517867
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.541591
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.530959
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.530865
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.512191
Train Epoch: 83 - Loss: 0.5185, Accuracy: 99.67%

Test set: Average loss: 0.5280, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.528760
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.541248
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.526846
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.522625
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.529885
Train Epoch: 84 - Loss: 0.5181, Accuracy: 99.68%

Test set: Average loss: 0.5280, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.510921
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.517589
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.518058
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.517413
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.507466
Train Epoch: 85 - Loss: 0.5178, Accuracy: 99.66%

Test set: Average loss: 0.5287, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.520751
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.522202
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.515205
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.519742
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.514990
Train Epoch: 86 - Loss: 0.5175, Accuracy: 99.69%

Test set: Average loss: 0.5277, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.537774
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.512891
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.507453
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.515979
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.509596
Train Epoch: 87 - Loss: 0.5172, Accuracy: 99.69%

Test set: Average loss: 0.5275, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.510282
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.511737
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.525578
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.509240
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.509052
Train Epoch: 88 - Loss: 0.5170, Accuracy: 99.69%

Test set: Average loss: 0.5273, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.510792
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.514849
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.515684
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.510145
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.507998
Train Epoch: 89 - Loss: 0.5167, Accuracy: 99.69%

Test set: Average loss: 0.5270, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.534534
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.509488
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.514906
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.526810
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.511517
Train Epoch: 90 - Loss: 0.5165, Accuracy: 99.71%

Test set: Average loss: 0.5273, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.509197
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.515684
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.515344
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.510097
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.515637
Train Epoch: 91 - Loss: 0.5163, Accuracy: 99.71%

Test set: Average loss: 0.5270, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.512714
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.509817
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.507236
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.528858
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.529215
Train Epoch: 92 - Loss: 0.5161, Accuracy: 99.73%

Test set: Average loss: 0.5267, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.506766
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.511294
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.538636
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.511759
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.510753
Train Epoch: 93 - Loss: 0.5160, Accuracy: 99.73%

Test set: Average loss: 0.5268, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.533988
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509462
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.509863
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.510108
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.530736
Train Epoch: 94 - Loss: 0.5158, Accuracy: 99.73%

Test set: Average loss: 0.5268, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.512421
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.516311
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.508332
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.506061
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.521913
Train Epoch: 95 - Loss: 0.5157, Accuracy: 99.73%

Test set: Average loss: 0.5268, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.514883
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.512795
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.508312
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508388
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.512308
Train Epoch: 96 - Loss: 0.5156, Accuracy: 99.73%

Test set: Average loss: 0.5267, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.508320
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509204
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.510047
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.513246
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.510688
Train Epoch: 97 - Loss: 0.5155, Accuracy: 99.73%

Test set: Average loss: 0.5267, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.520156
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.517468
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.519022
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.506806
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.524766
Train Epoch: 98 - Loss: 0.5155, Accuracy: 99.74%

Test set: Average loss: 0.5267, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.525395
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.524876
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.514055
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.537367
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.512472
Train Epoch: 99 - Loss: 0.5155, Accuracy: 99.74%

Test set: Average loss: 0.5267, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.512643
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.513922
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.511033
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.510794
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.515470
Train Epoch: 100 - Loss: 0.5154, Accuracy: 99.74%

Test set: Average loss: 0.5267, Accuracy: 9909/10000 (99.09%)


Training completed. Best Accuracy: 99.13%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.297549
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.153580
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.885307
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.852333
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.812783
Train Epoch: 1 - Loss: 1.0877, Accuracy: 80.36%

Test set: Average loss: 0.7544, Accuracy: 9234/10000 (92.34%)

New best model saved with accuracy: 92.34%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.752489
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.712363
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.710503
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.735664
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.680235
Train Epoch: 2 - Loss: 0.7167, Accuracy: 93.27%

Test set: Average loss: 0.6641, Accuracy: 9522/10000 (95.22%)

New best model saved with accuracy: 95.22%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.770732
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.611042
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.628325
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.626100
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.596644
Train Epoch: 3 - Loss: 0.6509, Accuracy: 95.57%

Test set: Average loss: 0.6289, Accuracy: 9638/10000 (96.38%)

New best model saved with accuracy: 96.38%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.631565
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.634289
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.601005
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.665001
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.603608
Train Epoch: 4 - Loss: 0.6185, Accuracy: 96.66%

Test set: Average loss: 0.5994, Accuracy: 9726/10000 (97.26%)

New best model saved with accuracy: 97.26%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.599613
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.644189
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.619272
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.551914
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.585488
Train Epoch: 5 - Loss: 0.5995, Accuracy: 97.30%

Test set: Average loss: 0.5866, Accuracy: 9757/10000 (97.57%)

New best model saved with accuracy: 97.57%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.596929
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.570216
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.608034
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.578609
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.591895
Train Epoch: 6 - Loss: 0.5868, Accuracy: 97.66%

Test set: Average loss: 0.5781, Accuracy: 9774/10000 (97.74%)

New best model saved with accuracy: 97.74%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.622462
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.631729
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.594148
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.616280
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.570557
Train Epoch: 7 - Loss: 0.5779, Accuracy: 97.87%

Test set: Average loss: 0.5723, Accuracy: 9780/10000 (97.80%)

New best model saved with accuracy: 97.80%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.571768
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.642450
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.593073
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.565008
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.540032
Train Epoch: 8 - Loss: 0.5709, Accuracy: 98.17%

Test set: Average loss: 0.5680, Accuracy: 9799/10000 (97.99%)

New best model saved with accuracy: 97.99%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.558858
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.576156
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.556307
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.534597
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.572773
Train Epoch: 9 - Loss: 0.5659, Accuracy: 98.28%

Test set: Average loss: 0.5638, Accuracy: 9828/10000 (98.28%)

New best model saved with accuracy: 98.28%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.579024
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.563499
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.573801
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.563087
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.572538
Train Epoch: 10 - Loss: 0.5618, Accuracy: 98.41%

Test set: Average loss: 0.5636, Accuracy: 9820/10000 (98.20%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.559523
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.544415
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.558399
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.547212
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.545865
Train Epoch: 11 - Loss: 0.5584, Accuracy: 98.55%

Test set: Average loss: 0.5573, Accuracy: 9835/10000 (98.35%)

New best model saved with accuracy: 98.35%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.566170
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.544030
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.537795
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.549582
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.555129
Train Epoch: 12 - Loss: 0.5557, Accuracy: 98.61%

Test set: Average loss: 0.5559, Accuracy: 9843/10000 (98.43%)

New best model saved with accuracy: 98.43%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.534299
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.550705
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.545268
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.571278
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.548766
Train Epoch: 13 - Loss: 0.5536, Accuracy: 98.67%

Test set: Average loss: 0.5547, Accuracy: 9839/10000 (98.39%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.547458
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.539522
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.548632
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.564062
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.550675
Train Epoch: 14 - Loss: 0.5518, Accuracy: 98.73%

Test set: Average loss: 0.5538, Accuracy: 9845/10000 (98.45%)

New best model saved with accuracy: 98.45%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.553949
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.529400
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.549680
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.574842
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.541881
Train Epoch: 15 - Loss: 0.5503, Accuracy: 98.77%

Test set: Average loss: 0.5534, Accuracy: 9852/10000 (98.52%)

New best model saved with accuracy: 98.52%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.537526
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.543781
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.529604
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.543661
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.529245
Train Epoch: 16 - Loss: 0.5491, Accuracy: 98.82%

Test set: Average loss: 0.5506, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.550018
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.562520
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.530153
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.526823
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.584693
Train Epoch: 17 - Loss: 0.5480, Accuracy: 98.86%

Test set: Average loss: 0.5503, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.537118
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.555664
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.558625
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.548064
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.541996
Train Epoch: 18 - Loss: 0.5472, Accuracy: 98.88%

Test set: Average loss: 0.5498, Accuracy: 9857/10000 (98.57%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.559244
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.563548
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.538726
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.533543
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.604337
Train Epoch: 19 - Loss: 0.5465, Accuracy: 98.88%

Test set: Average loss: 0.5493, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.552006
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.532440
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.567422
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.531535
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.532758
Train Epoch: 20 - Loss: 0.5460, Accuracy: 98.91%

Test set: Average loss: 0.5488, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.529470
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.530752
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.602876
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.537599
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.532767
Train Epoch: 21 - Loss: 0.5456, Accuracy: 98.94%

Test set: Average loss: 0.5487, Accuracy: 9864/10000 (98.64%)

New best model saved with accuracy: 98.64%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.543504
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.549645
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.547641
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.537508
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.561612
Train Epoch: 22 - Loss: 0.5453, Accuracy: 98.93%

Test set: Average loss: 0.5487, Accuracy: 9856/10000 (98.56%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.527145
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.543270
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.539881
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.567725
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.523057
Train Epoch: 23 - Loss: 0.5451, Accuracy: 98.94%

Test set: Average loss: 0.5484, Accuracy: 9860/10000 (98.60%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.570555
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.570277
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.577304
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.531271
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.552023
Train Epoch: 24 - Loss: 0.5450, Accuracy: 98.94%

Test set: Average loss: 0.5484, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.519754
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.547237
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.534624
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.561893
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.551136
Train Epoch: 25 - Loss: 0.5449, Accuracy: 98.95%

Test set: Average loss: 0.5484, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.559330
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.529282
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.549509
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.546708
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.534689
Train Epoch: 26 - Loss: 0.5472, Accuracy: 98.85%

Test set: Average loss: 0.5488, Accuracy: 9858/10000 (98.58%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.528544
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.516413
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.572176
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.552718
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.535373
Train Epoch: 27 - Loss: 0.5452, Accuracy: 98.94%

Test set: Average loss: 0.5460, Accuracy: 9864/10000 (98.64%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.542462
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.525921
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.551007
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.536451
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.561781
Train Epoch: 28 - Loss: 0.5433, Accuracy: 98.94%

Test set: Average loss: 0.5460, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.547011
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.556525
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.538525
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.525390
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.520841
Train Epoch: 29 - Loss: 0.5413, Accuracy: 99.00%

Test set: Average loss: 0.5442, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.566778
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.538374
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.565280
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.544130
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.525366
Train Epoch: 30 - Loss: 0.5397, Accuracy: 99.05%

Test set: Average loss: 0.5433, Accuracy: 9866/10000 (98.66%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.526961
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.549232
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.540475
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.535393
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.532922
Train Epoch: 31 - Loss: 0.5383, Accuracy: 99.09%

Test set: Average loss: 0.5428, Accuracy: 9872/10000 (98.72%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.535723
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.529969
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.525299
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.525520
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.551745
Train Epoch: 32 - Loss: 0.5370, Accuracy: 99.14%

Test set: Average loss: 0.5408, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.521165
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.526866
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.529385
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.545363
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.533129
Train Epoch: 33 - Loss: 0.5357, Accuracy: 99.21%

Test set: Average loss: 0.5394, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.520733
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.532575
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.537163
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.545802
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.536502
Train Epoch: 34 - Loss: 0.5344, Accuracy: 99.22%

Test set: Average loss: 0.5400, Accuracy: 9883/10000 (98.83%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.527507
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.528539
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.526123
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.537985
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.528140
Train Epoch: 35 - Loss: 0.5337, Accuracy: 99.26%

Test set: Average loss: 0.5389, Accuracy: 9878/10000 (98.78%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.530136
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.516096
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.524479
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.548274
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.530531
Train Epoch: 36 - Loss: 0.5328, Accuracy: 99.25%

Test set: Average loss: 0.5380, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.522827
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.535223
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.520564
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.535098
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.524578
Train Epoch: 37 - Loss: 0.5319, Accuracy: 99.29%

Test set: Average loss: 0.5374, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.514409
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.545637
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.516818
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.559486
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.552395
Train Epoch: 38 - Loss: 0.5312, Accuracy: 99.32%

Test set: Average loss: 0.5380, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.522518
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.524363
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.530357
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.509258
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.519034
Train Epoch: 39 - Loss: 0.5306, Accuracy: 99.34%

Test set: Average loss: 0.5373, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.547494
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.538607
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.558734
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.529876
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.526231
Train Epoch: 40 - Loss: 0.5301, Accuracy: 99.35%

Test set: Average loss: 0.5369, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.518365
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.524240
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.527288
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.529500
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.545003
Train Epoch: 41 - Loss: 0.5296, Accuracy: 99.36%

Test set: Average loss: 0.5364, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.515389
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.516795
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.519407
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.567115
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.535294
Train Epoch: 42 - Loss: 0.5291, Accuracy: 99.38%

Test set: Average loss: 0.5363, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.540654
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.530657
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.522135
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.527555
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.533569
Train Epoch: 43 - Loss: 0.5288, Accuracy: 99.40%

Test set: Average loss: 0.5362, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.522736
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.533089
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.514814
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.524463
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.529390
Train Epoch: 44 - Loss: 0.5285, Accuracy: 99.41%

Test set: Average loss: 0.5357, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.523775
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.532589
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.536027
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.532416
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.516657
Train Epoch: 45 - Loss: 0.5283, Accuracy: 99.41%

Test set: Average loss: 0.5357, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.542867
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.520506
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.519773
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.521638
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.530436
Train Epoch: 46 - Loss: 0.5281, Accuracy: 99.43%

Test set: Average loss: 0.5356, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.540340
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.522684
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.565413
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.512462
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.532398
Train Epoch: 47 - Loss: 0.5280, Accuracy: 99.43%

Test set: Average loss: 0.5357, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.518574
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.531508
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.560877
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.520971
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.534438
Train Epoch: 48 - Loss: 0.5278, Accuracy: 99.43%

Test set: Average loss: 0.5357, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.532392
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.519661
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.529560
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.530285
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.522370
Train Epoch: 49 - Loss: 0.5278, Accuracy: 99.44%

Test set: Average loss: 0.5356, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.518293
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.520485
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.533706
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.528250
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.544386
Train Epoch: 50 - Loss: 0.5277, Accuracy: 99.43%

Test set: Average loss: 0.5356, Accuracy: 9890/10000 (98.90%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.527586
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.525903
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.518530
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.523652
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.511939
Train Epoch: 51 - Loss: 0.5296, Accuracy: 99.35%

Test set: Average loss: 0.5361, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.530163
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.514756
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.529623
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.541621
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.525171
Train Epoch: 52 - Loss: 0.5289, Accuracy: 99.39%

Test set: Average loss: 0.5355, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.538570
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.517607
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.530904
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.530728
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.521311
Train Epoch: 53 - Loss: 0.5280, Accuracy: 99.40%

Test set: Average loss: 0.5353, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.533713
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.526629
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.523121
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.525318
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.532162
Train Epoch: 54 - Loss: 0.5272, Accuracy: 99.42%

Test set: Average loss: 0.5347, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.557214
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.534088
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.517473
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.544294
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.530810
Train Epoch: 55 - Loss: 0.5266, Accuracy: 99.41%

Test set: Average loss: 0.5362, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.515738
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.528547
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.517455
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.532947
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.521976
Train Epoch: 56 - Loss: 0.5260, Accuracy: 99.45%

Test set: Average loss: 0.5342, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.529953
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.515732
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.528995
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.518724
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.533905
Train Epoch: 57 - Loss: 0.5252, Accuracy: 99.50%

Test set: Average loss: 0.5331, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.536045
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.517320
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.519188
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.522057
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.539461
Train Epoch: 58 - Loss: 0.5246, Accuracy: 99.50%

Test set: Average loss: 0.5328, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.530734
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.517423
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.548128
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.519202
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.520866
Train Epoch: 59 - Loss: 0.5240, Accuracy: 99.52%

Test set: Average loss: 0.5343, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.519064
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.524966
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.513733
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.527196
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.516298
Train Epoch: 60 - Loss: 0.5235, Accuracy: 99.53%

Test set: Average loss: 0.5328, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.518882
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.521104
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.523964
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.509602
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.520820
Train Epoch: 61 - Loss: 0.5230, Accuracy: 99.54%

Test set: Average loss: 0.5321, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.529998
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.527235
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.527921
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.519438
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.525749
Train Epoch: 62 - Loss: 0.5225, Accuracy: 99.56%

Test set: Average loss: 0.5323, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.525999
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.524473
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.541716
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.519766
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.534157
Train Epoch: 63 - Loss: 0.5222, Accuracy: 99.55%

Test set: Average loss: 0.5314, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.527059
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.524102
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.517861
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.524097
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.512296
Train Epoch: 64 - Loss: 0.5217, Accuracy: 99.57%

Test set: Average loss: 0.5314, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.520496
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.561309
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.527792
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.517215
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.524227
Train Epoch: 65 - Loss: 0.5215, Accuracy: 99.58%

Test set: Average loss: 0.5312, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.526009
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.515376
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.527639
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.516367
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.516475
Train Epoch: 66 - Loss: 0.5212, Accuracy: 99.62%

Test set: Average loss: 0.5316, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.532755
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.536814
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.513013
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.522474
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.512215
Train Epoch: 67 - Loss: 0.5209, Accuracy: 99.61%

Test set: Average loss: 0.5307, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.509738
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.514751
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.509191
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.528693
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.523864
Train Epoch: 68 - Loss: 0.5207, Accuracy: 99.59%

Test set: Average loss: 0.5306, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.512378
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.512458
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.519496
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.511117
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.518325
Train Epoch: 69 - Loss: 0.5205, Accuracy: 99.62%

Test set: Average loss: 0.5305, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.508827
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.520996
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.520779
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.515494
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.516266
Train Epoch: 70 - Loss: 0.5203, Accuracy: 99.61%

Test set: Average loss: 0.5305, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.539930
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.523621
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.515201
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.514551
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.523371
Train Epoch: 71 - Loss: 0.5202, Accuracy: 99.63%

Test set: Average loss: 0.5305, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.520337
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.522521
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.511959
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.516219
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.511063
Train Epoch: 72 - Loss: 0.5201, Accuracy: 99.64%

Test set: Average loss: 0.5304, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.515105
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.514364
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.506604
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.521706
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.509215
Train Epoch: 73 - Loss: 0.5201, Accuracy: 99.64%

Test set: Average loss: 0.5304, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.522484
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.514714
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.525952
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.513295
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.514844
Train Epoch: 74 - Loss: 0.5200, Accuracy: 99.64%

Test set: Average loss: 0.5304, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.521283
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.514916
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.530440
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.525893
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.520166
Train Epoch: 75 - Loss: 0.5200, Accuracy: 99.64%

Test set: Average loss: 0.5304, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.523876
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.514675
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.536803
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.531079
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.534425
Train Epoch: 76 - Loss: 0.5216, Accuracy: 99.58%

Test set: Average loss: 0.5307, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.519432
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.530600
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.526406
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.543434
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.531031
Train Epoch: 77 - Loss: 0.5211, Accuracy: 99.61%

Test set: Average loss: 0.5319, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.528883
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.514262
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.519454
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.509900
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.512645
Train Epoch: 78 - Loss: 0.5206, Accuracy: 99.60%

Test set: Average loss: 0.5304, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.526438
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.508846
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.514191
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.514199
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.516615
Train Epoch: 79 - Loss: 0.5202, Accuracy: 99.61%

Test set: Average loss: 0.5307, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.509689
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.507686
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.515918
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.513205
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.521744
Train Epoch: 80 - Loss: 0.5198, Accuracy: 99.63%

Test set: Average loss: 0.5314, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.513846
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.535809
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.534804
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.515485
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.524207
Train Epoch: 81 - Loss: 0.5193, Accuracy: 99.64%

Test set: Average loss: 0.5299, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.536627
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.518922
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.518090
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.524918
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.510416
Train Epoch: 82 - Loss: 0.5189, Accuracy: 99.65%

Test set: Average loss: 0.5291, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.520458
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.523049
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.508180
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.540545
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.520403
Train Epoch: 83 - Loss: 0.5185, Accuracy: 99.65%

Test set: Average loss: 0.5300, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.516525
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.522041
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.516037
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.524923
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.526357
Train Epoch: 84 - Loss: 0.5181, Accuracy: 99.69%

Test set: Average loss: 0.5293, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.514952
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.515241
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.517363
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.523735
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.512472
Train Epoch: 85 - Loss: 0.5178, Accuracy: 99.70%

Test set: Average loss: 0.5289, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.509393
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.514761
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.516941
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.524086
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.518871
Train Epoch: 86 - Loss: 0.5174, Accuracy: 99.71%

Test set: Average loss: 0.5286, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.536561
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.521222
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.515359
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.509620
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.530728
Train Epoch: 87 - Loss: 0.5172, Accuracy: 99.70%

Test set: Average loss: 0.5282, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.513164
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.511773
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.508208
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.539728
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.515493
Train Epoch: 88 - Loss: 0.5169, Accuracy: 99.71%

Test set: Average loss: 0.5282, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.506658
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.520345
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.520269
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.530432
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.516261
Train Epoch: 89 - Loss: 0.5166, Accuracy: 99.71%

Test set: Average loss: 0.5290, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.515606
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.507626
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.511359
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.521475
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.515402
Train Epoch: 90 - Loss: 0.5164, Accuracy: 99.73%

Test set: Average loss: 0.5281, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.513000
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.514799
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.542812
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.507256
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.512214
Train Epoch: 91 - Loss: 0.5161, Accuracy: 99.74%

Test set: Average loss: 0.5280, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.513077
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.516052
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510752
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.510626
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.511621
Train Epoch: 92 - Loss: 0.5159, Accuracy: 99.74%

Test set: Average loss: 0.5282, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.511756
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.512682
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.511905
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.523268
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.527636
Train Epoch: 93 - Loss: 0.5158, Accuracy: 99.74%

Test set: Average loss: 0.5279, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.513377
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509377
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.511574
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.508457
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.523886
Train Epoch: 94 - Loss: 0.5157, Accuracy: 99.76%

Test set: Average loss: 0.5277, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.507882
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.510094
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.519138
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.511506
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509945
Train Epoch: 95 - Loss: 0.5156, Accuracy: 99.75%

Test set: Average loss: 0.5278, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.507949
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.505624
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.517569
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.517912
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.508013
Train Epoch: 96 - Loss: 0.5155, Accuracy: 99.76%

Test set: Average loss: 0.5278, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.539946
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508473
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.514497
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.527675
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.517591
Train Epoch: 97 - Loss: 0.5154, Accuracy: 99.76%

Test set: Average loss: 0.5278, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.517773
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.512384
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.522177
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511604
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.520052
Train Epoch: 98 - Loss: 0.5153, Accuracy: 99.77%

Test set: Average loss: 0.5278, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.514782
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.514097
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.511485
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.508520
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.509241
Train Epoch: 99 - Loss: 0.5153, Accuracy: 99.77%

Test set: Average loss: 0.5277, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.510919
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.511321
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.525626
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.510084
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.510905
Train Epoch: 100 - Loss: 0.5153, Accuracy: 99.77%

Test set: Average loss: 0.5277, Accuracy: 9910/10000 (99.10%)


Training completed. Best Accuracy: 99.12%
