Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.645385
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.905827
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.687219
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.681010
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.672196
Train Epoch: 1 - Loss: 0.8110, Accuracy: 89.10%

Test set: Average loss: 0.6136, Accuracy: 9731/10000 (97.31%)

New best model saved with accuracy: 97.31%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.616713
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.587584
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.561810
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.583433
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.603267
Train Epoch: 2 - Loss: 0.6028, Accuracy: 97.47%

Test set: Average loss: 0.5887, Accuracy: 9796/10000 (97.96%)

New best model saved with accuracy: 97.96%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.645692
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.618950
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.578717
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.553871
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.576664
Train Epoch: 3 - Loss: 0.5781, Accuracy: 98.12%

Test set: Average loss: 0.5666, Accuracy: 9845/10000 (98.45%)

New best model saved with accuracy: 98.45%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.574319
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.557718
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.581881
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.557902
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.535845
Train Epoch: 4 - Loss: 0.5649, Accuracy: 98.53%

Test set: Average loss: 0.5609, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.563631
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.562995
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.570834
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.574035
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.551208
Train Epoch: 5 - Loss: 0.5576, Accuracy: 98.67%

Test set: Average loss: 0.5541, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.554933
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.528835
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.550192
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.563723
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.546692
Train Epoch: 6 - Loss: 0.5518, Accuracy: 98.83%

Test set: Average loss: 0.5468, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.532784
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.573032
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.541436
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.532680
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.536174
Train Epoch: 7 - Loss: 0.5475, Accuracy: 98.96%

Test set: Average loss: 0.5465, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.538575
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.526876
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.533244
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.537762
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.571710
Train Epoch: 8 - Loss: 0.5442, Accuracy: 99.03%

Test set: Average loss: 0.5430, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.534949
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.547557
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.527477
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.557600
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.550056
Train Epoch: 9 - Loss: 0.5415, Accuracy: 99.14%

Test set: Average loss: 0.5413, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.552022
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.521939
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.547889
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.529308
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.524010
Train Epoch: 10 - Loss: 0.5389, Accuracy: 99.19%

Test set: Average loss: 0.5434, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.554922
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.535930
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.532335
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.518200
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.527689
Train Epoch: 11 - Loss: 0.5371, Accuracy: 99.27%

Test set: Average loss: 0.5396, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.540056
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.523619
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.525963
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.532660
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.552286
Train Epoch: 12 - Loss: 0.5356, Accuracy: 99.29%

Test set: Average loss: 0.5384, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.531768
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.522474
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.570800
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.530283
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.530922
Train Epoch: 13 - Loss: 0.5342, Accuracy: 99.33%

Test set: Average loss: 0.5381, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.533881
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.565612
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.536208
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.538692
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.540828
Train Epoch: 14 - Loss: 0.5328, Accuracy: 99.36%

Test set: Average loss: 0.5368, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.519247
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.520394
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.539977
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.547927
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.523228
Train Epoch: 15 - Loss: 0.5318, Accuracy: 99.37%

Test set: Average loss: 0.5351, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.542753
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.521774
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.515791
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.520532
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.551575
Train Epoch: 16 - Loss: 0.5309, Accuracy: 99.41%

Test set: Average loss: 0.5347, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.517570
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.525262
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.541725
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.537594
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.536668
Train Epoch: 17 - Loss: 0.5302, Accuracy: 99.43%

Test set: Average loss: 0.5351, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.521334
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.520371
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.537388
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.518441
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.523773
Train Epoch: 18 - Loss: 0.5296, Accuracy: 99.45%

Test set: Average loss: 0.5345, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.520954
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.519629
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.532054
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.527333
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.526657
Train Epoch: 19 - Loss: 0.5290, Accuracy: 99.48%

Test set: Average loss: 0.5342, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.538924
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.516107
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.532452
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.520656
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.515918
Train Epoch: 20 - Loss: 0.5286, Accuracy: 99.47%

Test set: Average loss: 0.5339, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.539033
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.520178
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.520441
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.521918
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.515819
Train Epoch: 21 - Loss: 0.5282, Accuracy: 99.49%

Test set: Average loss: 0.5333, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.530473
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.527777
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.515289
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.532271
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.515789
Train Epoch: 22 - Loss: 0.5280, Accuracy: 99.50%

Test set: Average loss: 0.5331, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.523682
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.525709
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.523562
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.527142
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.530771
Train Epoch: 23 - Loss: 0.5278, Accuracy: 99.51%

Test set: Average loss: 0.5330, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.528875
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.521996
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.561477
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.550340
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.534767
Train Epoch: 24 - Loss: 0.5277, Accuracy: 99.51%

Test set: Average loss: 0.5330, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.537144
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.517098
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.515242
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.526891
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.524522
Train Epoch: 25 - Loss: 0.5276, Accuracy: 99.51%

Test set: Average loss: 0.5330, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.520885
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.565303
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.536455
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.521804
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.515244
Train Epoch: 26 - Loss: 0.5338, Accuracy: 99.31%

Test set: Average loss: 0.5442, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.552651
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.526279
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.526206
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.533650
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.520892
Train Epoch: 27 - Loss: 0.5317, Accuracy: 99.38%

Test set: Average loss: 0.5331, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.545609
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.528261
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.520358
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.527005
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.530099
Train Epoch: 28 - Loss: 0.5304, Accuracy: 99.39%

Test set: Average loss: 0.5332, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.523695
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.530798
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.524545
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.525019
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.518580
Train Epoch: 29 - Loss: 0.5295, Accuracy: 99.43%

Test set: Average loss: 0.5322, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.524059
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.515306
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.544293
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.554859
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.514435
Train Epoch: 30 - Loss: 0.5282, Accuracy: 99.47%

Test set: Average loss: 0.5321, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.528433
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.528874
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.540546
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.553328
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.525629
Train Epoch: 31 - Loss: 0.5270, Accuracy: 99.52%

Test set: Average loss: 0.5331, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.515050
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.514250
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.521709
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.526341
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.522775
Train Epoch: 32 - Loss: 0.5261, Accuracy: 99.53%

Test set: Average loss: 0.5325, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.524702
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.523019
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.519076
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.519625
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.532040
Train Epoch: 33 - Loss: 0.5252, Accuracy: 99.53%

Test set: Average loss: 0.5338, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.526029
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.516693
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.518694
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.522285
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.524772
Train Epoch: 34 - Loss: 0.5242, Accuracy: 99.56%

Test set: Average loss: 0.5301, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.514672
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.535328
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.529524
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.521696
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.532012
Train Epoch: 35 - Loss: 0.5234, Accuracy: 99.59%

Test set: Average loss: 0.5299, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.520031
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.543764
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.527842
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.519437
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.511978
Train Epoch: 36 - Loss: 0.5228, Accuracy: 99.62%

Test set: Average loss: 0.5285, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.531249
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.514957
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.522804
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.516991
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.521193
Train Epoch: 37 - Loss: 0.5220, Accuracy: 99.63%

Test set: Average loss: 0.5288, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.530388
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.522167
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.532496
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.512358
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.527073
Train Epoch: 38 - Loss: 0.5214, Accuracy: 99.64%

Test set: Average loss: 0.5290, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.512771
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.514325
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.522979
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.516202
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.513864
Train Epoch: 39 - Loss: 0.5209, Accuracy: 99.66%

Test set: Average loss: 0.5281, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.508909
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.508326
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.521312
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.514382
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.517333
Train Epoch: 40 - Loss: 0.5204, Accuracy: 99.67%

Test set: Average loss: 0.5282, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.530817
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.521545
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.521467
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.513687
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.517836
Train Epoch: 41 - Loss: 0.5199, Accuracy: 99.70%

Test set: Average loss: 0.5273, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.531162
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.526032
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.515563
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.527758
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.510167
Train Epoch: 42 - Loss: 0.5196, Accuracy: 99.69%

Test set: Average loss: 0.5270, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.513297
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.522457
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.517376
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.518024
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.515241
Train Epoch: 43 - Loss: 0.5193, Accuracy: 99.72%

Test set: Average loss: 0.5271, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.520316
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.516508
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.516522
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.516601
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.516856
Train Epoch: 44 - Loss: 0.5189, Accuracy: 99.74%

Test set: Average loss: 0.5267, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.510526
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.532404
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.511591
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.517105
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.514113
Train Epoch: 45 - Loss: 0.5187, Accuracy: 99.74%

Test set: Average loss: 0.5268, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.515617
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.517540
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.522180
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.511077
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.517529
Train Epoch: 46 - Loss: 0.5186, Accuracy: 99.74%

Test set: Average loss: 0.5267, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.520056
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.516151
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.529201
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.511698
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.517033
Train Epoch: 47 - Loss: 0.5184, Accuracy: 99.75%

Test set: Average loss: 0.5266, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.526505
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.514021
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.512446
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.508775
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.521292
Train Epoch: 48 - Loss: 0.5183, Accuracy: 99.75%

Test set: Average loss: 0.5266, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.519122
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.518090
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.513540
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.518154
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.512613
Train Epoch: 49 - Loss: 0.5182, Accuracy: 99.76%

Test set: Average loss: 0.5266, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.517411
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.520745
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.523746
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.519813
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.517026
Train Epoch: 50 - Loss: 0.5182, Accuracy: 99.76%

Test set: Average loss: 0.5266, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.510303
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.517412
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.517456
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.516268
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.515041
Train Epoch: 51 - Loss: 0.5219, Accuracy: 99.64%

Test set: Average loss: 0.5288, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.521551
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.541601
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.518635
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.532499
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.511261
Train Epoch: 52 - Loss: 0.5214, Accuracy: 99.65%

Test set: Average loss: 0.5294, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.515394
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.511437
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.511930
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.542051
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.525860
Train Epoch: 53 - Loss: 0.5206, Accuracy: 99.67%

Test set: Average loss: 0.5277, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.528006
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.519617
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.512111
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.518040
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.518868
Train Epoch: 54 - Loss: 0.5203, Accuracy: 99.67%

Test set: Average loss: 0.5280, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.522687
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.542998
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.519664
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.523412
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.512559
Train Epoch: 55 - Loss: 0.5196, Accuracy: 99.67%

Test set: Average loss: 0.5262, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.525236
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.528630
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.515615
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.517640
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.518765
Train Epoch: 56 - Loss: 0.5191, Accuracy: 99.71%

Test set: Average loss: 0.5264, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.512144
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.538146
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.520732
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.514050
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.523645
Train Epoch: 57 - Loss: 0.5185, Accuracy: 99.73%

Test set: Average loss: 0.5287, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.515465
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.521494
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.513466
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.513155
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.517097
Train Epoch: 58 - Loss: 0.5178, Accuracy: 99.74%

Test set: Average loss: 0.5256, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.514960
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.513483
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.509846
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.510664
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.520181
Train Epoch: 59 - Loss: 0.5173, Accuracy: 99.77%

Test set: Average loss: 0.5260, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.514065
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.518077
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.518877
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.526629
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.523183
Train Epoch: 60 - Loss: 0.5168, Accuracy: 99.76%

Test set: Average loss: 0.5253, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.512282
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.512828
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.522273
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.518994
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.522647
Train Epoch: 61 - Loss: 0.5164, Accuracy: 99.79%

Test set: Average loss: 0.5257, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.512248
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.515127
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.533454
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.512047
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.528366
Train Epoch: 62 - Loss: 0.5161, Accuracy: 99.80%

Test set: Average loss: 0.5246, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.515580
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.514045
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.511308
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.521660
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.516767
Train Epoch: 63 - Loss: 0.5156, Accuracy: 99.81%

Test set: Average loss: 0.5251, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.518166
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.510519
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.506648
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.515974
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.512052
Train Epoch: 64 - Loss: 0.5153, Accuracy: 99.83%

Test set: Average loss: 0.5244, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.509643
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.512717
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.532230
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.508766
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.515635
Train Epoch: 65 - Loss: 0.5149, Accuracy: 99.83%

Test set: Average loss: 0.5241, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.517797
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.515299
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.515626
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.512114
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.511675
Train Epoch: 66 - Loss: 0.5147, Accuracy: 99.85%

Test set: Average loss: 0.5240, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.523609
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.509467
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.510941
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.510988
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513132
Train Epoch: 67 - Loss: 0.5145, Accuracy: 99.84%

Test set: Average loss: 0.5238, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.515336
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.508237
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.514053
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.514287
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.511038
Train Epoch: 68 - Loss: 0.5142, Accuracy: 99.85%

Test set: Average loss: 0.5243, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.512894
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.513872
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.540616
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.508423
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.513782
Train Epoch: 69 - Loss: 0.5140, Accuracy: 99.86%

Test set: Average loss: 0.5239, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.508455
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.516535
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.508972
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.511037
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.522970
Train Epoch: 70 - Loss: 0.5139, Accuracy: 99.85%

Test set: Average loss: 0.5239, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.512329
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.522393
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.518033
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.511191
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.511565
Train Epoch: 71 - Loss: 0.5137, Accuracy: 99.86%

Test set: Average loss: 0.5237, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.509770
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509372
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.518245
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.510846
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.511166
Train Epoch: 72 - Loss: 0.5136, Accuracy: 99.86%

Test set: Average loss: 0.5238, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.510499
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.515051
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.508515
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.510827
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.510144
Train Epoch: 73 - Loss: 0.5136, Accuracy: 99.86%

Test set: Average loss: 0.5237, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.512588
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.518059
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.515828
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.507254
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.514548
Train Epoch: 74 - Loss: 0.5135, Accuracy: 99.87%

Test set: Average loss: 0.5237, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.507512
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.509699
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.512018
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.511530
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.512582
Train Epoch: 75 - Loss: 0.5135, Accuracy: 99.87%

Test set: Average loss: 0.5237, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.513765
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.533502
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.537165
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.514588
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.526965
Train Epoch: 76 - Loss: 0.5167, Accuracy: 99.77%

Test set: Average loss: 0.5252, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.512025
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.520106
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.511592
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.511296
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.515370
Train Epoch: 77 - Loss: 0.5161, Accuracy: 99.81%

Test set: Average loss: 0.5249, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.510840
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.524802
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.511799
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.508722
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.521711
Train Epoch: 78 - Loss: 0.5158, Accuracy: 99.78%

Test set: Average loss: 0.5241, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.510162
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.510637
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.507834
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.516023
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.509572
Train Epoch: 79 - Loss: 0.5152, Accuracy: 99.81%

Test set: Average loss: 0.5258, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.509727
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.523840
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.509717
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.524941
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.511226
Train Epoch: 80 - Loss: 0.5149, Accuracy: 99.80%

Test set: Average loss: 0.5246, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.522966
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.511844
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.517894
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.516632
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.510389
Train Epoch: 81 - Loss: 0.5147, Accuracy: 99.80%

Test set: Average loss: 0.5241, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.509637
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.508266
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.512591
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.537521
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.516416
Train Epoch: 82 - Loss: 0.5142, Accuracy: 99.84%

Test set: Average loss: 0.5240, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.517447
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.507265
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.512282
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.509695
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.511674
Train Epoch: 83 - Loss: 0.5139, Accuracy: 99.83%

Test set: Average loss: 0.5230, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.513460
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.509348
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.507859
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.510230
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.511407
Train Epoch: 84 - Loss: 0.5133, Accuracy: 99.86%

Test set: Average loss: 0.5240, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.515681
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.515863
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.511331
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.506176
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.517951
Train Epoch: 85 - Loss: 0.5131, Accuracy: 99.86%

Test set: Average loss: 0.5231, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.511741
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.508119
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.513018
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.511030
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.512461
Train Epoch: 86 - Loss: 0.5128, Accuracy: 99.87%

Test set: Average loss: 0.5248, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.514244
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.510607
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.506097
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.515929
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.514385
Train Epoch: 87 - Loss: 0.5124, Accuracy: 99.87%

Test set: Average loss: 0.5227, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.537079
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.510683
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.521294
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.506109
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.512107
Train Epoch: 88 - Loss: 0.5122, Accuracy: 99.88%

Test set: Average loss: 0.5227, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.516544
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.528381
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.513668
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.517325
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.506898
Train Epoch: 89 - Loss: 0.5119, Accuracy: 99.89%

Test set: Average loss: 0.5233, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.514049
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.512354
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.506961
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.510401
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.508898
Train Epoch: 90 - Loss: 0.5117, Accuracy: 99.89%

Test set: Average loss: 0.5225, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.510008
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.507399
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.508838
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.507428
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.517042
Train Epoch: 91 - Loss: 0.5115, Accuracy: 99.90%

Test set: Average loss: 0.5224, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.508755
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.507385
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.512666
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507073
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.514362
Train Epoch: 92 - Loss: 0.5113, Accuracy: 99.91%

Test set: Average loss: 0.5226, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.510173
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.506107
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.508399
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.514965
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.507475
Train Epoch: 93 - Loss: 0.5112, Accuracy: 99.91%

Test set: Average loss: 0.5224, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.510750
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.512846
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.515489
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.509076
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.510356
Train Epoch: 94 - Loss: 0.5110, Accuracy: 99.91%

Test set: Average loss: 0.5222, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.512907
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.508074
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.508183
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.514986
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.510791
Train Epoch: 95 - Loss: 0.5109, Accuracy: 99.91%

Test set: Average loss: 0.5222, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.533952
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.517852
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.528489
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.513738
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.514567
Train Epoch: 96 - Loss: 0.5108, Accuracy: 99.91%

Test set: Average loss: 0.5222, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.511957
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.512405
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.511536
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.511824
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507979
Train Epoch: 97 - Loss: 0.5108, Accuracy: 99.92%

Test set: Average loss: 0.5221, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.513994
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.510144
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508915
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511384
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.505756
Train Epoch: 98 - Loss: 0.5107, Accuracy: 99.92%

Test set: Average loss: 0.5221, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506703
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.511785
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506825
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.511518
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.516653
Train Epoch: 99 - Loss: 0.5107, Accuracy: 99.92%

Test set: Average loss: 0.5221, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.531778
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.510440
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.510415
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.508466
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.510022
Train Epoch: 100 - Loss: 0.5106, Accuracy: 99.92%

Test set: Average loss: 0.5221, Accuracy: 9936/10000 (99.36%)


Training completed. Best Accuracy: 99.41%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.641729
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.802008
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.793201
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.715989
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.645423
Train Epoch: 1 - Loss: 0.8581, Accuracy: 86.80%

Test set: Average loss: 0.6200, Accuracy: 9694/10000 (96.94%)

New best model saved with accuracy: 96.94%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.584612
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.606263
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.603387
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.616169
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.594230
Train Epoch: 2 - Loss: 0.6010, Accuracy: 97.39%

Test set: Average loss: 0.5842, Accuracy: 9791/10000 (97.91%)

New best model saved with accuracy: 97.91%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.603683
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.569288
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.602915
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.594777
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.568001
Train Epoch: 3 - Loss: 0.5772, Accuracy: 98.14%

Test set: Average loss: 0.5907, Accuracy: 9744/10000 (97.44%)

Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.611936
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.608513
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.567026
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.548914
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.598845
Train Epoch: 4 - Loss: 0.5651, Accuracy: 98.49%

Test set: Average loss: 0.5558, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.537828
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.554313
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.585706
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.540836
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.553828
Train Epoch: 5 - Loss: 0.5573, Accuracy: 98.67%

Test set: Average loss: 0.5524, Accuracy: 9870/10000 (98.70%)

Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.541672
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.535446
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.546806
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.543016
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.533076
Train Epoch: 6 - Loss: 0.5516, Accuracy: 98.82%

Test set: Average loss: 0.5556, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.554340
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.570781
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.587317
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.560233
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.540044
Train Epoch: 7 - Loss: 0.5473, Accuracy: 98.98%

Test set: Average loss: 0.5456, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.548550
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.534483
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.535940
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.551001
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.567608
Train Epoch: 8 - Loss: 0.5438, Accuracy: 99.04%

Test set: Average loss: 0.5420, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.523993
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.565249
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.544350
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.562579
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.538133
Train Epoch: 9 - Loss: 0.5410, Accuracy: 99.14%

Test set: Average loss: 0.5406, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.524575
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.534468
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.541561
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.537371
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.543161
Train Epoch: 10 - Loss: 0.5388, Accuracy: 99.16%

Test set: Average loss: 0.5390, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.527428
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.536005
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.589099
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.534338
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.550103
Train Epoch: 11 - Loss: 0.5367, Accuracy: 99.24%

Test set: Average loss: 0.5373, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.515591
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.549661
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.542830
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.573916
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.534594
Train Epoch: 12 - Loss: 0.5352, Accuracy: 99.28%

Test set: Average loss: 0.5358, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.545118
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.542118
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.528626
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.517510
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.521497
Train Epoch: 13 - Loss: 0.5336, Accuracy: 99.31%

Test set: Average loss: 0.5356, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.535604
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.518024
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.518537
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.516420
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.559642
Train Epoch: 14 - Loss: 0.5324, Accuracy: 99.36%

Test set: Average loss: 0.5351, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.529167
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.523426
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.539228
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.530889
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.529454
Train Epoch: 15 - Loss: 0.5313, Accuracy: 99.38%

Test set: Average loss: 0.5344, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.540944
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.525378
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.553291
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.522271
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.532798
Train Epoch: 16 - Loss: 0.5305, Accuracy: 99.41%

Test set: Average loss: 0.5342, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.529308
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.513236
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.525839
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.527297
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.522479
Train Epoch: 17 - Loss: 0.5297, Accuracy: 99.44%

Test set: Average loss: 0.5329, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.516101
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.531660
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.525201
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.522325
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.526512
Train Epoch: 18 - Loss: 0.5290, Accuracy: 99.45%

Test set: Average loss: 0.5334, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.529875
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.516649
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.526655
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.552513
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.547794
Train Epoch: 19 - Loss: 0.5285, Accuracy: 99.48%

Test set: Average loss: 0.5325, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.532158
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.561509
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.519856
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.527485
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.514914
Train Epoch: 20 - Loss: 0.5281, Accuracy: 99.50%

Test set: Average loss: 0.5323, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.526696
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.523470
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.522522
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.512006
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.515220
Train Epoch: 21 - Loss: 0.5277, Accuracy: 99.50%

Test set: Average loss: 0.5323, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.531976
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.513190
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.518403
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.518535
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.517219
Train Epoch: 22 - Loss: 0.5275, Accuracy: 99.51%

Test set: Average loss: 0.5319, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.542725
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.517451
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.535164
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.522816
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.523976
Train Epoch: 23 - Loss: 0.5273, Accuracy: 99.53%

Test set: Average loss: 0.5320, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.517369
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.532124
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.526589
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.517904
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.526977
Train Epoch: 24 - Loss: 0.5271, Accuracy: 99.53%

Test set: Average loss: 0.5319, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.538172
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.515334
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.524562
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.519244
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.525056
Train Epoch: 25 - Loss: 0.5271, Accuracy: 99.53%

Test set: Average loss: 0.5319, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.518152
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.546564
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.544154
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.516634
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.523565
Train Epoch: 26 - Loss: 0.5327, Accuracy: 99.33%

Test set: Average loss: 0.5344, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.525230
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.545618
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.530397
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.519461
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.521585
Train Epoch: 27 - Loss: 0.5311, Accuracy: 99.37%

Test set: Average loss: 0.5330, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.566326
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.526533
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.530846
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.543145
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.549597
Train Epoch: 28 - Loss: 0.5297, Accuracy: 99.40%

Test set: Average loss: 0.5326, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.520428
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.525575
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.532390
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.513320
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.535732
Train Epoch: 29 - Loss: 0.5282, Accuracy: 99.44%

Test set: Average loss: 0.5305, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.525617
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.557155
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.518358
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.529997
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.525174
Train Epoch: 30 - Loss: 0.5273, Accuracy: 99.47%

Test set: Average loss: 0.5304, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.522840
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.531723
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.518610
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.517196
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.516127
Train Epoch: 31 - Loss: 0.5263, Accuracy: 99.47%

Test set: Average loss: 0.5315, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.518543
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.517091
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.514004
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.526455
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.533186
Train Epoch: 32 - Loss: 0.5252, Accuracy: 99.54%

Test set: Average loss: 0.5291, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.512306
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.521458
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.524072
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.514597
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.522290
Train Epoch: 33 - Loss: 0.5243, Accuracy: 99.58%

Test set: Average loss: 0.5284, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.523911
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.512211
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.528870
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.515419
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.520404
Train Epoch: 34 - Loss: 0.5234, Accuracy: 99.58%

Test set: Average loss: 0.5287, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.528623
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.513595
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.534364
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.518329
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.522909
Train Epoch: 35 - Loss: 0.5227, Accuracy: 99.61%

Test set: Average loss: 0.5297, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.524485
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.509018
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.509176
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.530384
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.534817
Train Epoch: 36 - Loss: 0.5217, Accuracy: 99.64%

Test set: Average loss: 0.5289, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.513677
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.516866
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.535470
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.530996
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.515179
Train Epoch: 37 - Loss: 0.5211, Accuracy: 99.65%

Test set: Average loss: 0.5297, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.526513
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.519905
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.521572
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.537785
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.525598
Train Epoch: 38 - Loss: 0.5206, Accuracy: 99.67%

Test set: Average loss: 0.5278, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.509786
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.530782
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.512906
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.511867
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.513649
Train Epoch: 39 - Loss: 0.5200, Accuracy: 99.69%

Test set: Average loss: 0.5267, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.519328
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.518655
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.513766
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.516806
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.537627
Train Epoch: 40 - Loss: 0.5195, Accuracy: 99.69%

Test set: Average loss: 0.5261, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.525710
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.515088
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.513151
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.518684
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.520883
Train Epoch: 41 - Loss: 0.5191, Accuracy: 99.73%

Test set: Average loss: 0.5256, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.524553
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.510681
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.507647
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.522414
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.525467
Train Epoch: 42 - Loss: 0.5188, Accuracy: 99.73%

Test set: Average loss: 0.5257, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.512197
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.511627
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.524386
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.517658
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.509832
Train Epoch: 43 - Loss: 0.5184, Accuracy: 99.74%

Test set: Average loss: 0.5258, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.509134
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.512492
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.512827
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.512137
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.515776
Train Epoch: 44 - Loss: 0.5181, Accuracy: 99.73%

Test set: Average loss: 0.5256, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.515208
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.513721
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.521239
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.514023
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.527719
Train Epoch: 45 - Loss: 0.5179, Accuracy: 99.74%

Test set: Average loss: 0.5253, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.508430
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.520295
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.528507
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.515562
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.511039
Train Epoch: 46 - Loss: 0.5177, Accuracy: 99.75%

Test set: Average loss: 0.5253, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.510629
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.535878
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.514757
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.509889
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.539630
Train Epoch: 47 - Loss: 0.5176, Accuracy: 99.76%

Test set: Average loss: 0.5251, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.510387
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.513147
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.519449
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.513179
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.513492
Train Epoch: 48 - Loss: 0.5175, Accuracy: 99.77%

Test set: Average loss: 0.5251, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.522729
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.514576
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.515013
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.512453
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.519012
Train Epoch: 49 - Loss: 0.5174, Accuracy: 99.77%

Test set: Average loss: 0.5251, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.518743
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.513216
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.520663
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.516202
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.512947
Train Epoch: 50 - Loss: 0.5173, Accuracy: 99.76%

Test set: Average loss: 0.5250, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.513534
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.520201
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.520714
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.523708
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.514509
Train Epoch: 51 - Loss: 0.5211, Accuracy: 99.64%

Test set: Average loss: 0.5290, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.515411
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.509337
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.527254
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.511885
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.512956
Train Epoch: 52 - Loss: 0.5204, Accuracy: 99.65%

Test set: Average loss: 0.5267, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.514987
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.515782
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.514068
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.520025
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.516063
Train Epoch: 53 - Loss: 0.5199, Accuracy: 99.67%

Test set: Average loss: 0.5287, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.529433
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.523984
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.512871
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.513978
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.507354
Train Epoch: 54 - Loss: 0.5193, Accuracy: 99.70%

Test set: Average loss: 0.5272, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.512151
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.516027
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.522853
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.515994
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.526998
Train Epoch: 55 - Loss: 0.5187, Accuracy: 99.70%

Test set: Average loss: 0.5259, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.512111
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.519757
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.513997
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.527241
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.529233
Train Epoch: 56 - Loss: 0.5180, Accuracy: 99.73%

Test set: Average loss: 0.5251, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.532107
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.512280
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.512275
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.515317
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.511712
Train Epoch: 57 - Loss: 0.5177, Accuracy: 99.73%

Test set: Average loss: 0.5244, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.511093
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.513605
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.520893
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.510074
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.520788
Train Epoch: 58 - Loss: 0.5171, Accuracy: 99.73%

Test set: Average loss: 0.5236, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.516078
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.509408
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.517629
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.512750
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.513743
Train Epoch: 59 - Loss: 0.5166, Accuracy: 99.75%

Test set: Average loss: 0.5238, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.510778
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.513028
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.513219
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.516270
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.512102
Train Epoch: 60 - Loss: 0.5161, Accuracy: 99.78%

Test set: Average loss: 0.5241, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.513274
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.519993
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.517331
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.538027
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.518875
Train Epoch: 61 - Loss: 0.5158, Accuracy: 99.79%

Test set: Average loss: 0.5245, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.517853
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.509518
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.516131
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.508891
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.514139
Train Epoch: 62 - Loss: 0.5152, Accuracy: 99.79%

Test set: Average loss: 0.5238, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.512467
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.512066
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.514284
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.512138
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.515209
Train Epoch: 63 - Loss: 0.5149, Accuracy: 99.80%

Test set: Average loss: 0.5234, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.513235
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.514396
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.517727
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.516292
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.518310
Train Epoch: 64 - Loss: 0.5146, Accuracy: 99.81%

Test set: Average loss: 0.5230, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.520440
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.512639
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.516710
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.507435
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.513094
Train Epoch: 65 - Loss: 0.5142, Accuracy: 99.83%

Test set: Average loss: 0.5230, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.508794
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.514883
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.508855
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.511197
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.523202
Train Epoch: 66 - Loss: 0.5139, Accuracy: 99.83%

Test set: Average loss: 0.5230, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.510804
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.517210
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.510797
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.511585
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.507654
Train Epoch: 67 - Loss: 0.5137, Accuracy: 99.83%

Test set: Average loss: 0.5229, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.525127
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.510882
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.513162
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.514718
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.512548
Train Epoch: 68 - Loss: 0.5135, Accuracy: 99.85%

Test set: Average loss: 0.5229, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.521188
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.513014
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.510590
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.510070
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.511769
Train Epoch: 69 - Loss: 0.5133, Accuracy: 99.86%

Test set: Average loss: 0.5227, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.511470
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.509088
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.515882
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.514831
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.509512
Train Epoch: 70 - Loss: 0.5132, Accuracy: 99.85%

Test set: Average loss: 0.5225, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.511310
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.509060
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.507685
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.519688
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.512355
Train Epoch: 71 - Loss: 0.5130, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.509372
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.511563
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.519581
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.508795
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.524726
Train Epoch: 72 - Loss: 0.5130, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.506694
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.511853
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.513796
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.508218
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.509373
Train Epoch: 73 - Loss: 0.5129, Accuracy: 99.86%

Test set: Average loss: 0.5225, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.509905
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.514598
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.510283
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.508835
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.518293
Train Epoch: 74 - Loss: 0.5128, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.508686
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.507104
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.514401
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.508126
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.506660
Train Epoch: 75 - Loss: 0.5128, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.515030
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.535096
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.517799
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.512938
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.516891
Train Epoch: 76 - Loss: 0.5155, Accuracy: 99.78%

Test set: Average loss: 0.5258, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.517554
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.535898
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.522122
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.514183
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.517120
Train Epoch: 77 - Loss: 0.5152, Accuracy: 99.79%

Test set: Average loss: 0.5261, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.515724
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.522141
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.510585
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.518086
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.512645
Train Epoch: 78 - Loss: 0.5150, Accuracy: 99.78%

Test set: Average loss: 0.5233, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.506422
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.514891
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.526196
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.509836
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.508164
Train Epoch: 79 - Loss: 0.5147, Accuracy: 99.80%

Test set: Average loss: 0.5238, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.508563
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.512190
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.510500
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.509063
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.511923
Train Epoch: 80 - Loss: 0.5142, Accuracy: 99.80%

Test set: Average loss: 0.5234, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.508638
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.509232
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.508779
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.523401
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.510833
Train Epoch: 81 - Loss: 0.5138, Accuracy: 99.81%

Test set: Average loss: 0.5238, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.520541
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.509432
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.510967
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.510583
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.514538
Train Epoch: 82 - Loss: 0.5135, Accuracy: 99.84%

Test set: Average loss: 0.5224, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.516441
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512137
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.510853
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.506457
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.507618
Train Epoch: 83 - Loss: 0.5129, Accuracy: 99.84%

Test set: Average loss: 0.5227, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.509244
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.507829
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.511960
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.509994
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.509200
Train Epoch: 84 - Loss: 0.5128, Accuracy: 99.84%

Test set: Average loss: 0.5219, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508744
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.512293
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.506492
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.508804
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.508924
Train Epoch: 85 - Loss: 0.5123, Accuracy: 99.85%

Test set: Average loss: 0.5220, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.509119
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.507391
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.506773
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.528288
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.508676
Train Epoch: 86 - Loss: 0.5120, Accuracy: 99.88%

Test set: Average loss: 0.5216, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.511677
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.508743
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.509124
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.514868
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.513918
Train Epoch: 87 - Loss: 0.5118, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.513563
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.508164
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.507108
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.507686
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.511453
Train Epoch: 88 - Loss: 0.5115, Accuracy: 99.88%

Test set: Average loss: 0.5218, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.505885
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.506826
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.510905
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.515250
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.513384
Train Epoch: 89 - Loss: 0.5113, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.506409
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.522362
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.508219
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.509103
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.507198
Train Epoch: 90 - Loss: 0.5111, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.511746
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.507325
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509057
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.507688
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.508479
Train Epoch: 91 - Loss: 0.5109, Accuracy: 99.90%

Test set: Average loss: 0.5213, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.509833
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.513145
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.511878
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.508083
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.507214
Train Epoch: 92 - Loss: 0.5107, Accuracy: 99.89%

Test set: Average loss: 0.5219, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.508803
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.508701
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.507459
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.515427
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.521946
Train Epoch: 93 - Loss: 0.5105, Accuracy: 99.90%

Test set: Average loss: 0.5210, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.510323
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.513944
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.505504
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.505610
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.506572
Train Epoch: 94 - Loss: 0.5104, Accuracy: 99.91%

Test set: Average loss: 0.5210, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.506771
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.508854
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506243
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.510137
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509950
Train Epoch: 95 - Loss: 0.5103, Accuracy: 99.91%

Test set: Average loss: 0.5209, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.513487
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.525250
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.513246
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508924
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.506070
Train Epoch: 96 - Loss: 0.5102, Accuracy: 99.90%

Test set: Average loss: 0.5209, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.505061
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509752
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.511756
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.512139
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.509715
Train Epoch: 97 - Loss: 0.5101, Accuracy: 99.91%

Test set: Average loss: 0.5209, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.507884
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.506792
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.506593
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.507838
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.507965
Train Epoch: 98 - Loss: 0.5101, Accuracy: 99.91%

Test set: Average loss: 0.5209, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.511344
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.514663
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.509477
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.506300
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.506439
Train Epoch: 99 - Loss: 0.5101, Accuracy: 99.91%

Test set: Average loss: 0.5209, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.506255
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.527338
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.509336
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.505238
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.506305
Train Epoch: 100 - Loss: 0.5100, Accuracy: 99.91%

Test set: Average loss: 0.5209, Accuracy: 9943/10000 (99.43%)


Training completed. Best Accuracy: 99.46%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.899447
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.055685
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.747624
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.633399
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.631016
Train Epoch: 1 - Loss: 0.9388, Accuracy: 82.88%

Test set: Average loss: 0.6291, Accuracy: 9644/10000 (96.44%)

New best model saved with accuracy: 96.44%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.628564
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.623606
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.598506
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.614007
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.579070
Train Epoch: 2 - Loss: 0.6064, Accuracy: 97.34%

Test set: Average loss: 0.5805, Accuracy: 9812/10000 (98.12%)

New best model saved with accuracy: 98.12%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.587227
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.618194
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.574249
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.570221
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.580022
Train Epoch: 3 - Loss: 0.5790, Accuracy: 98.03%

Test set: Average loss: 0.5702, Accuracy: 9827/10000 (98.27%)

New best model saved with accuracy: 98.27%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.558331
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.580638
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.551781
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.561066
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.539171
Train Epoch: 4 - Loss: 0.5656, Accuracy: 98.39%

Test set: Average loss: 0.5683, Accuracy: 9827/10000 (98.27%)

Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.560514
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.591081
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.568629
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.535514
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.538653
Train Epoch: 5 - Loss: 0.5575, Accuracy: 98.61%

Test set: Average loss: 0.5527, Accuracy: 9866/10000 (98.66%)

New best model saved with accuracy: 98.66%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.559378
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.587241
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.536627
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.566316
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.574107
Train Epoch: 6 - Loss: 0.5516, Accuracy: 98.82%

Test set: Average loss: 0.5507, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.556499
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.569387
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.527075
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.573268
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.528567
Train Epoch: 7 - Loss: 0.5470, Accuracy: 98.97%

Test set: Average loss: 0.5451, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.541537
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.543172
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.530005
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.529312
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.536681
Train Epoch: 8 - Loss: 0.5434, Accuracy: 99.02%

Test set: Average loss: 0.5430, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.537136
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.558407
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.522945
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.549548
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.517243
Train Epoch: 9 - Loss: 0.5407, Accuracy: 99.09%

Test set: Average loss: 0.5399, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.523296
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.546368
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.525210
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.525881
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.528492
Train Epoch: 10 - Loss: 0.5385, Accuracy: 99.14%

Test set: Average loss: 0.5400, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.566951
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.549178
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.539258
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.526695
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.527242
Train Epoch: 11 - Loss: 0.5365, Accuracy: 99.22%

Test set: Average loss: 0.5377, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.528419
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.516880
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.540901
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.531320
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.539172
Train Epoch: 12 - Loss: 0.5348, Accuracy: 99.24%

Test set: Average loss: 0.5368, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.518565
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.547364
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.527335
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.532634
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.547910
Train Epoch: 13 - Loss: 0.5332, Accuracy: 99.31%

Test set: Average loss: 0.5360, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.521953
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.547784
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.524081
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.519173
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.544796
Train Epoch: 14 - Loss: 0.5321, Accuracy: 99.34%

Test set: Average loss: 0.5349, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.523836
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.524073
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.525204
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.531682
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.539974
Train Epoch: 15 - Loss: 0.5310, Accuracy: 99.39%

Test set: Average loss: 0.5345, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.517152
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.534819
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.532363
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.531133
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.555949
Train Epoch: 16 - Loss: 0.5302, Accuracy: 99.40%

Test set: Average loss: 0.5348, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.530071
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.536981
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.519395
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.523532
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.538548
Train Epoch: 17 - Loss: 0.5294, Accuracy: 99.44%

Test set: Average loss: 0.5337, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.524721
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.528470
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.516094
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.528512
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.540095
Train Epoch: 18 - Loss: 0.5288, Accuracy: 99.44%

Test set: Average loss: 0.5340, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.522813
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.525955
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.550539
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.530189
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.515732
Train Epoch: 19 - Loss: 0.5283, Accuracy: 99.46%

Test set: Average loss: 0.5332, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.517507
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.538791
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.516064
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.541243
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.532158
Train Epoch: 20 - Loss: 0.5278, Accuracy: 99.47%

Test set: Average loss: 0.5329, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.518390
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.525220
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.514317
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.527370
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.517047
Train Epoch: 21 - Loss: 0.5275, Accuracy: 99.49%

Test set: Average loss: 0.5327, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.536483
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.532809
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.520415
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.556809
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.523132
Train Epoch: 22 - Loss: 0.5272, Accuracy: 99.50%

Test set: Average loss: 0.5325, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.528072
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.520259
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.518696
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.508961
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.521450
Train Epoch: 23 - Loss: 0.5271, Accuracy: 99.51%

Test set: Average loss: 0.5325, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.523905
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.522853
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.529136
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.527395
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.525288
Train Epoch: 24 - Loss: 0.5269, Accuracy: 99.52%

Test set: Average loss: 0.5325, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.558584
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.547303
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.540847
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.528633
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.527927
Train Epoch: 25 - Loss: 0.5269, Accuracy: 99.51%

Test set: Average loss: 0.5325, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.520839
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.517809
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.543482
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.524203
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.535421
Train Epoch: 26 - Loss: 0.5324, Accuracy: 99.33%

Test set: Average loss: 0.5363, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.524792
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.524827
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.527289
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.523093
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.542234
Train Epoch: 27 - Loss: 0.5307, Accuracy: 99.37%

Test set: Average loss: 0.5352, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.531927
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.519027
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.535623
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.519308
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.515299
Train Epoch: 28 - Loss: 0.5290, Accuracy: 99.44%

Test set: Average loss: 0.5334, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.523695
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.526792
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.517084
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.530737
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.535808
Train Epoch: 29 - Loss: 0.5280, Accuracy: 99.46%

Test set: Average loss: 0.5317, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.517622
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.515432
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.520358
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.535086
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.511663
Train Epoch: 30 - Loss: 0.5270, Accuracy: 99.47%

Test set: Average loss: 0.5320, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.519780
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.525183
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.517339
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.528242
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.531765
Train Epoch: 31 - Loss: 0.5259, Accuracy: 99.52%

Test set: Average loss: 0.5308, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.518044
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.533445
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.526612
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.521572
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.522370
Train Epoch: 32 - Loss: 0.5250, Accuracy: 99.52%

Test set: Average loss: 0.5295, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.520018
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.530466
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.522667
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.525754
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.513856
Train Epoch: 33 - Loss: 0.5240, Accuracy: 99.53%

Test set: Average loss: 0.5314, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.523298
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.516108
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.512381
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.524472
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.531127
Train Epoch: 34 - Loss: 0.5232, Accuracy: 99.58%

Test set: Average loss: 0.5319, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.515713
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.523486
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.518159
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.514881
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.511772
Train Epoch: 35 - Loss: 0.5225, Accuracy: 99.60%

Test set: Average loss: 0.5289, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.510056
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.526109
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.534301
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.532869
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.510613
Train Epoch: 36 - Loss: 0.5217, Accuracy: 99.63%

Test set: Average loss: 0.5287, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.522828
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.552249
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.527972
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.523064
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.519093
Train Epoch: 37 - Loss: 0.5210, Accuracy: 99.65%

Test set: Average loss: 0.5293, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.514381
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.509162
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.515422
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.528969
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.511776
Train Epoch: 38 - Loss: 0.5205, Accuracy: 99.65%

Test set: Average loss: 0.5277, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.515196
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.520669
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.526110
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.522942
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.515096
Train Epoch: 39 - Loss: 0.5200, Accuracy: 99.66%

Test set: Average loss: 0.5275, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.510873
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.512691
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.516349
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.521197
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.510397
Train Epoch: 40 - Loss: 0.5196, Accuracy: 99.70%

Test set: Average loss: 0.5289, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.538011
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.519463
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.533387
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.526380
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.518298
Train Epoch: 41 - Loss: 0.5191, Accuracy: 99.71%

Test set: Average loss: 0.5268, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.516328
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.521259
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.524583
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.509373
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.521901
Train Epoch: 42 - Loss: 0.5187, Accuracy: 99.72%

Test set: Average loss: 0.5265, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.516648
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.522305
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.519104
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.517289
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.509511
Train Epoch: 43 - Loss: 0.5184, Accuracy: 99.73%

Test set: Average loss: 0.5270, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.513592
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.517432
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.511113
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.514297
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.510306
Train Epoch: 44 - Loss: 0.5182, Accuracy: 99.73%

Test set: Average loss: 0.5264, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.520919
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.521304
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.511044
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.518111
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.507507
Train Epoch: 45 - Loss: 0.5179, Accuracy: 99.75%

Test set: Average loss: 0.5264, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.513740
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.520591
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.516085
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.512905
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.514435
Train Epoch: 46 - Loss: 0.5177, Accuracy: 99.76%

Test set: Average loss: 0.5263, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.535769
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.522070
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.514247
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.514832
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.522969
Train Epoch: 47 - Loss: 0.5176, Accuracy: 99.76%

Test set: Average loss: 0.5262, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.524445
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.524015
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.513352
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.517967
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.512711
Train Epoch: 48 - Loss: 0.5175, Accuracy: 99.77%

Test set: Average loss: 0.5261, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.515963
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.508775
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.511507
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.512075
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.511858
Train Epoch: 49 - Loss: 0.5174, Accuracy: 99.77%

Test set: Average loss: 0.5261, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.517873
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.514163
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.518771
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.518098
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.517397
Train Epoch: 50 - Loss: 0.5174, Accuracy: 99.77%

Test set: Average loss: 0.5261, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.513080
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.522562
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.526959
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.522624
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.521769
Train Epoch: 51 - Loss: 0.5214, Accuracy: 99.63%

Test set: Average loss: 0.5267, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.520302
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.512660
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.524075
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.518995
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.520121
Train Epoch: 52 - Loss: 0.5204, Accuracy: 99.64%

Test set: Average loss: 0.5272, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.514063
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.512770
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.512687
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.520443
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.532497
Train Epoch: 53 - Loss: 0.5199, Accuracy: 99.66%

Test set: Average loss: 0.5263, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.511529
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.516900
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.515339
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.517164
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.524652
Train Epoch: 54 - Loss: 0.5193, Accuracy: 99.67%

Test set: Average loss: 0.5275, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.512943
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.509602
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.511732
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.515513
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.548242
Train Epoch: 55 - Loss: 0.5186, Accuracy: 99.70%

Test set: Average loss: 0.5262, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.509809
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.520562
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.513931
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.515443
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.517293
Train Epoch: 56 - Loss: 0.5181, Accuracy: 99.71%

Test set: Average loss: 0.5257, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.520416
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.517124
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.520901
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.515065
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.519450
Train Epoch: 57 - Loss: 0.5175, Accuracy: 99.75%

Test set: Average loss: 0.5268, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.528052
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.511278
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.510776
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.512522
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.514390
Train Epoch: 58 - Loss: 0.5171, Accuracy: 99.76%

Test set: Average loss: 0.5269, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.516651
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.536283
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.511105
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.511821
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.516160
Train Epoch: 59 - Loss: 0.5168, Accuracy: 99.75%

Test set: Average loss: 0.5251, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.510692
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.515044
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.516041
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.509826
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.513456
Train Epoch: 60 - Loss: 0.5162, Accuracy: 99.78%

Test set: Average loss: 0.5258, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.510201
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.510839
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.510917
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.515565
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.519710
Train Epoch: 61 - Loss: 0.5157, Accuracy: 99.80%

Test set: Average loss: 0.5241, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.511170
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.510844
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.509857
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.536520
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.507188
Train Epoch: 62 - Loss: 0.5154, Accuracy: 99.81%

Test set: Average loss: 0.5247, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.511286
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.508343
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.511580
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.510879
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.524502
Train Epoch: 63 - Loss: 0.5150, Accuracy: 99.80%

Test set: Average loss: 0.5247, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.513420
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.508944
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.517180
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.508876
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.509039
Train Epoch: 64 - Loss: 0.5146, Accuracy: 99.82%

Test set: Average loss: 0.5243, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.510050
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.512188
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.513360
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.511250
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.539172
Train Epoch: 65 - Loss: 0.5144, Accuracy: 99.83%

Test set: Average loss: 0.5242, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.510988
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.507138
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.508022
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.532220
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.509890
Train Epoch: 66 - Loss: 0.5141, Accuracy: 99.84%

Test set: Average loss: 0.5241, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.531913
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.515346
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.505384
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.508695
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513666
Train Epoch: 67 - Loss: 0.5138, Accuracy: 99.85%

Test set: Average loss: 0.5238, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.515079
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.519063
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.507634
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.510446
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.514316
Train Epoch: 68 - Loss: 0.5136, Accuracy: 99.86%

Test set: Average loss: 0.5240, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.526086
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.520326
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.514456
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.529022
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.530946
Train Epoch: 69 - Loss: 0.5134, Accuracy: 99.86%

Test set: Average loss: 0.5234, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.511779
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.509843
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.507818
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.510232
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.511690
Train Epoch: 70 - Loss: 0.5133, Accuracy: 99.86%

Test set: Average loss: 0.5235, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.519322
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.508468
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.514007
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.528245
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.512594
Train Epoch: 71 - Loss: 0.5131, Accuracy: 99.87%

Test set: Average loss: 0.5235, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.512002
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509840
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.507700
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.513919
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.508063
Train Epoch: 72 - Loss: 0.5131, Accuracy: 99.87%

Test set: Average loss: 0.5234, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.507033
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.526641
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.514502
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.513097
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.507910
Train Epoch: 73 - Loss: 0.5130, Accuracy: 99.88%

Test set: Average loss: 0.5234, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.508579
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.507855
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.510769
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.513132
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.510839
Train Epoch: 74 - Loss: 0.5129, Accuracy: 99.88%

Test set: Average loss: 0.5234, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.510572
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.524272
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.516437
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.511029
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.508730
Train Epoch: 75 - Loss: 0.5129, Accuracy: 99.88%

Test set: Average loss: 0.5234, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.509312
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.510024
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.518223
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.519248
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.516231
Train Epoch: 76 - Loss: 0.5157, Accuracy: 99.78%

Test set: Average loss: 0.5253, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.521617
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.512841
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.522738
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.508965
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.520893
Train Epoch: 77 - Loss: 0.5154, Accuracy: 99.79%

Test set: Average loss: 0.5245, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.515174
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.513146
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.511735
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.516003
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.512446
Train Epoch: 78 - Loss: 0.5151, Accuracy: 99.78%

Test set: Average loss: 0.5241, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.509858
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.509451
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.511187
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.515614
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.509212
Train Epoch: 79 - Loss: 0.5145, Accuracy: 99.81%

Test set: Average loss: 0.5253, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.517262
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.513357
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.514007
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.516446
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.531861
Train Epoch: 80 - Loss: 0.5143, Accuracy: 99.81%

Test set: Average loss: 0.5237, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.509455
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.517181
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.510015
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.512344
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.511850
Train Epoch: 81 - Loss: 0.5139, Accuracy: 99.82%

Test set: Average loss: 0.5239, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.514550
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.513054
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.510474
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.515075
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.507892
Train Epoch: 82 - Loss: 0.5134, Accuracy: 99.83%

Test set: Average loss: 0.5229, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.508617
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.522590
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.505590
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.510785
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.514822
Train Epoch: 83 - Loss: 0.5132, Accuracy: 99.83%

Test set: Average loss: 0.5235, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.509650
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.517494
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.509854
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.507724
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.544562
Train Epoch: 84 - Loss: 0.5128, Accuracy: 99.86%

Test set: Average loss: 0.5245, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.515673
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.513999
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.507517
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.507746
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.509649
Train Epoch: 85 - Loss: 0.5124, Accuracy: 99.87%

Test set: Average loss: 0.5228, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.517504
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.533716
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.515749
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.507873
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.506388
Train Epoch: 86 - Loss: 0.5122, Accuracy: 99.88%

Test set: Average loss: 0.5224, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.512522
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.522800
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.516459
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.513181
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.516212
Train Epoch: 87 - Loss: 0.5119, Accuracy: 99.88%

Test set: Average loss: 0.5233, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.507578
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.508442
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.515257
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.509016
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.507683
Train Epoch: 88 - Loss: 0.5117, Accuracy: 99.88%

Test set: Average loss: 0.5230, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.507003
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.529438
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.512543
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.507615
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.517629
Train Epoch: 89 - Loss: 0.5114, Accuracy: 99.89%

Test set: Average loss: 0.5224, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.512684
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.509186
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.521331
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508139
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.513607
Train Epoch: 90 - Loss: 0.5112, Accuracy: 99.90%

Test set: Average loss: 0.5223, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.508129
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.509336
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.505574
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.515348
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.510053
Train Epoch: 91 - Loss: 0.5110, Accuracy: 99.90%

Test set: Average loss: 0.5221, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.508940
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.507981
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510994
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507419
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.509046
Train Epoch: 92 - Loss: 0.5109, Accuracy: 99.91%

Test set: Average loss: 0.5222, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.508492
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.506033
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.505934
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.511924
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.508266
Train Epoch: 93 - Loss: 0.5107, Accuracy: 99.91%

Test set: Average loss: 0.5221, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.506868
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.513512
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.507532
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.509812
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.517598
Train Epoch: 94 - Loss: 0.5105, Accuracy: 99.91%

Test set: Average loss: 0.5219, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.505815
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.506187
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.510569
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.507714
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.507548
Train Epoch: 95 - Loss: 0.5104, Accuracy: 99.92%

Test set: Average loss: 0.5220, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.509868
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.511078
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507570
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.512094
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.504672
Train Epoch: 96 - Loss: 0.5104, Accuracy: 99.91%

Test set: Average loss: 0.5219, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.511339
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.510638
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.515958
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.509577
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507635
Train Epoch: 97 - Loss: 0.5103, Accuracy: 99.92%

Test set: Average loss: 0.5219, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.508440
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.513870
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.507579
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508379
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.506269
Train Epoch: 98 - Loss: 0.5102, Accuracy: 99.92%

Test set: Average loss: 0.5219, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506674
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.508589
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.518867
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.514459
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.508794
Train Epoch: 99 - Loss: 0.5102, Accuracy: 99.92%

Test set: Average loss: 0.5219, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.510593
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.509018
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.509599
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.512550
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.508819
Train Epoch: 100 - Loss: 0.5102, Accuracy: 99.92%

Test set: Average loss: 0.5219, Accuracy: 9929/10000 (99.29%)


Training completed. Best Accuracy: 99.37%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.421173
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.947408
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.737924
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.722423
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.697253
Train Epoch: 1 - Loss: 0.8580, Accuracy: 86.97%

Test set: Average loss: 0.6269, Accuracy: 9681/10000 (96.81%)

New best model saved with accuracy: 96.81%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.654518
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.579659
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.599445
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.553666
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.628902
Train Epoch: 2 - Loss: 0.6085, Accuracy: 97.27%

Test set: Average loss: 0.5968, Accuracy: 9755/10000 (97.55%)

New best model saved with accuracy: 97.55%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.610854
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.565943
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.568971
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.556842
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.583796
Train Epoch: 3 - Loss: 0.5821, Accuracy: 98.00%

Test set: Average loss: 0.5726, Accuracy: 9822/10000 (98.22%)

New best model saved with accuracy: 98.22%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.578310
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.558903
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.562227
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.545544
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.539297
Train Epoch: 4 - Loss: 0.5688, Accuracy: 98.38%

Test set: Average loss: 0.5647, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.585758
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.554161
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.560825
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.590000
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.580602
Train Epoch: 5 - Loss: 0.5606, Accuracy: 98.61%

Test set: Average loss: 0.5619, Accuracy: 9840/10000 (98.40%)

Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.538563
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.539168
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.549014
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.536391
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.545505
Train Epoch: 6 - Loss: 0.5545, Accuracy: 98.72%

Test set: Average loss: 0.5525, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.554638
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.553256
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.555808
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.573318
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.588600
Train Epoch: 7 - Loss: 0.5503, Accuracy: 98.84%

Test set: Average loss: 0.5518, Accuracy: 9875/10000 (98.75%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.538984
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.538547
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.560577
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.552375
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.526212
Train Epoch: 8 - Loss: 0.5467, Accuracy: 98.95%

Test set: Average loss: 0.5459, Accuracy: 9890/10000 (98.90%)

New best model saved with accuracy: 98.90%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.550833
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.523980
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.536654
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.551982
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.523489
Train Epoch: 9 - Loss: 0.5436, Accuracy: 99.05%

Test set: Average loss: 0.5462, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.542810
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.544156
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.534533
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.538925
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.537522
Train Epoch: 10 - Loss: 0.5413, Accuracy: 99.09%

Test set: Average loss: 0.5414, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.522794
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.526035
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.526651
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.549503
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.530837
Train Epoch: 11 - Loss: 0.5390, Accuracy: 99.19%

Test set: Average loss: 0.5397, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.531050
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.524009
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.526132
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.541366
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.521218
Train Epoch: 12 - Loss: 0.5373, Accuracy: 99.19%

Test set: Average loss: 0.5409, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.537318
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.527071
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.546333
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.533563
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.543178
Train Epoch: 13 - Loss: 0.5357, Accuracy: 99.28%

Test set: Average loss: 0.5380, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.521502
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.534454
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.534312
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.554457
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.530938
Train Epoch: 14 - Loss: 0.5345, Accuracy: 99.29%

Test set: Average loss: 0.5375, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.540015
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.539041
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.534101
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.534471
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.551691
Train Epoch: 15 - Loss: 0.5334, Accuracy: 99.34%

Test set: Average loss: 0.5364, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.528753
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.540814
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.525318
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.526230
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.521295
Train Epoch: 16 - Loss: 0.5324, Accuracy: 99.34%

Test set: Average loss: 0.5362, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.536918
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.530306
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.554566
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.538138
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.548452
Train Epoch: 17 - Loss: 0.5316, Accuracy: 99.34%

Test set: Average loss: 0.5359, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.544878
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.517056
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.537364
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.521031
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.539350
Train Epoch: 18 - Loss: 0.5310, Accuracy: 99.38%

Test set: Average loss: 0.5349, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.552872
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.538010
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.536059
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.528267
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.529176
Train Epoch: 19 - Loss: 0.5304, Accuracy: 99.40%

Test set: Average loss: 0.5349, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.515523
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.526513
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.537573
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.522743
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.545016
Train Epoch: 20 - Loss: 0.5299, Accuracy: 99.42%

Test set: Average loss: 0.5347, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.521650
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.515603
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.530997
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.523720
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.526388
Train Epoch: 21 - Loss: 0.5296, Accuracy: 99.43%

Test set: Average loss: 0.5343, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.523976
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.524187
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.519902
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.532669
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.523026
Train Epoch: 22 - Loss: 0.5293, Accuracy: 99.44%

Test set: Average loss: 0.5344, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.518687
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.520601
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.545800
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.534712
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.553958
Train Epoch: 23 - Loss: 0.5291, Accuracy: 99.45%

Test set: Average loss: 0.5343, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.527800
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.526765
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.533733
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.539238
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.533480
Train Epoch: 24 - Loss: 0.5290, Accuracy: 99.46%

Test set: Average loss: 0.5342, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.529456
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.523374
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.521000
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.523396
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.526839
Train Epoch: 25 - Loss: 0.5289, Accuracy: 99.46%

Test set: Average loss: 0.5342, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.542473
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.544700
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.564709
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.513933
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.533307
Train Epoch: 26 - Loss: 0.5349, Accuracy: 99.25%

Test set: Average loss: 0.5357, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.525955
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.528058
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.524488
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.522346
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.523170
Train Epoch: 27 - Loss: 0.5328, Accuracy: 99.33%

Test set: Average loss: 0.5362, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.521208
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.530488
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.518660
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.542776
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.518450
Train Epoch: 28 - Loss: 0.5315, Accuracy: 99.36%

Test set: Average loss: 0.5343, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.559319
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.521350
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.522300
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.543320
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.526871
Train Epoch: 29 - Loss: 0.5301, Accuracy: 99.38%

Test set: Average loss: 0.5402, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.542537
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.528631
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.529631
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.521892
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.519120
Train Epoch: 30 - Loss: 0.5291, Accuracy: 99.41%

Test set: Average loss: 0.5360, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.520144
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.539763
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.525336
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.527463
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.524100
Train Epoch: 31 - Loss: 0.5280, Accuracy: 99.45%

Test set: Average loss: 0.5340, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.527407
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.531642
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.524975
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.518261
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.519213
Train Epoch: 32 - Loss: 0.5267, Accuracy: 99.46%

Test set: Average loss: 0.5326, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.522531
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.531706
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.521301
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.516762
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.522936
Train Epoch: 33 - Loss: 0.5258, Accuracy: 99.49%

Test set: Average loss: 0.5352, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.535472
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.523782
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.514513
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.523255
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.533674
Train Epoch: 34 - Loss: 0.5252, Accuracy: 99.50%

Test set: Average loss: 0.5305, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.519303
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.519505
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.562493
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.525589
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.530650
Train Epoch: 35 - Loss: 0.5241, Accuracy: 99.55%

Test set: Average loss: 0.5299, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.524065
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.547060
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.519769
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.518781
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.543149
Train Epoch: 36 - Loss: 0.5235, Accuracy: 99.58%

Test set: Average loss: 0.5292, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.511159
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.522355
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.517700
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.515089
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.529758
Train Epoch: 37 - Loss: 0.5227, Accuracy: 99.57%

Test set: Average loss: 0.5289, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.516786
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.511266
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.523380
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.520926
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.519875
Train Epoch: 38 - Loss: 0.5220, Accuracy: 99.61%

Test set: Average loss: 0.5305, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.511373
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.520380
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.517818
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.524421
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.512119
Train Epoch: 39 - Loss: 0.5215, Accuracy: 99.64%

Test set: Average loss: 0.5286, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.521615
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.519699
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.515479
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.524583
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.520054
Train Epoch: 40 - Loss: 0.5210, Accuracy: 99.63%

Test set: Average loss: 0.5280, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.519968
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.518101
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.520766
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.519425
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.512781
Train Epoch: 41 - Loss: 0.5206, Accuracy: 99.66%

Test set: Average loss: 0.5277, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.522906
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.515744
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.512889
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.517267
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.517869
Train Epoch: 42 - Loss: 0.5202, Accuracy: 99.66%

Test set: Average loss: 0.5277, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.513140
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.527584
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.514276
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.514538
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.512476
Train Epoch: 43 - Loss: 0.5198, Accuracy: 99.67%

Test set: Average loss: 0.5274, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.512371
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.527363
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.510474
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.533317
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.516643
Train Epoch: 44 - Loss: 0.5195, Accuracy: 99.71%

Test set: Average loss: 0.5272, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.523338
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.539035
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.518701
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.513401
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.520811
Train Epoch: 45 - Loss: 0.5193, Accuracy: 99.69%

Test set: Average loss: 0.5274, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.515850
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.511609
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.517707
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.535030
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.515446
Train Epoch: 46 - Loss: 0.5191, Accuracy: 99.70%

Test set: Average loss: 0.5271, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.508825
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.524066
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.533107
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.524638
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.521579
Train Epoch: 47 - Loss: 0.5190, Accuracy: 99.71%

Test set: Average loss: 0.5273, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.554603
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.528025
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.520352
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.515519
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.524460
Train Epoch: 48 - Loss: 0.5189, Accuracy: 99.71%

Test set: Average loss: 0.5271, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.516040
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.535908
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.517191
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.518730
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.515225
Train Epoch: 49 - Loss: 0.5188, Accuracy: 99.71%

Test set: Average loss: 0.5271, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.513556
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.514706
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.512478
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.510886
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.511001
Train Epoch: 50 - Loss: 0.5187, Accuracy: 99.71%

Test set: Average loss: 0.5271, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.522992
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.531924
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.540072
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.513808
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.522938
Train Epoch: 51 - Loss: 0.5225, Accuracy: 99.58%

Test set: Average loss: 0.5287, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.531105
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.526044
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.534377
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.522158
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.515954
Train Epoch: 52 - Loss: 0.5218, Accuracy: 99.60%

Test set: Average loss: 0.5288, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.524129
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.535920
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.522357
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.520602
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.538235
Train Epoch: 53 - Loss: 0.5212, Accuracy: 99.63%

Test set: Average loss: 0.5278, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.519342
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.511377
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.515613
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.517590
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.517955
Train Epoch: 54 - Loss: 0.5206, Accuracy: 99.67%

Test set: Average loss: 0.5289, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.524035
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.522103
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.510188
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.517268
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.530155
Train Epoch: 55 - Loss: 0.5200, Accuracy: 99.65%

Test set: Average loss: 0.5293, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.522477
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.530829
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.515182
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.516816
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.525837
Train Epoch: 56 - Loss: 0.5198, Accuracy: 99.67%

Test set: Average loss: 0.5262, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.518789
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.524632
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.520609
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.537976
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.534884
Train Epoch: 57 - Loss: 0.5191, Accuracy: 99.67%

Test set: Average loss: 0.5258, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.513368
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.522814
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.521945
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.521440
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.525751
Train Epoch: 58 - Loss: 0.5183, Accuracy: 99.71%

Test set: Average loss: 0.5260, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.513805
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.521767
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.520039
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.515763
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.515292
Train Epoch: 59 - Loss: 0.5179, Accuracy: 99.72%

Test set: Average loss: 0.5269, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.515888
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.517817
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.524824
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.516355
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.512251
Train Epoch: 60 - Loss: 0.5173, Accuracy: 99.73%

Test set: Average loss: 0.5253, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.513347
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.519915
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.511755
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.511563
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.512567
Train Epoch: 61 - Loss: 0.5170, Accuracy: 99.76%

Test set: Average loss: 0.5260, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.511867
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.523274
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.522380
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.515172
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.528375
Train Epoch: 62 - Loss: 0.5165, Accuracy: 99.76%

Test set: Average loss: 0.5250, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.512955
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.509096
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.521357
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.517321
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.511550
Train Epoch: 63 - Loss: 0.5162, Accuracy: 99.77%

Test set: Average loss: 0.5248, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.511904
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.509802
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.546061
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.520635
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.512826
Train Epoch: 64 - Loss: 0.5157, Accuracy: 99.79%

Test set: Average loss: 0.5246, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.511856
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.513233
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.516083
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.516899
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.509991
Train Epoch: 65 - Loss: 0.5155, Accuracy: 99.77%

Test set: Average loss: 0.5240, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.516538
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.509215
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.513815
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.511940
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.519133
Train Epoch: 66 - Loss: 0.5151, Accuracy: 99.80%

Test set: Average loss: 0.5247, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.516176
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.513357
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.516521
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.508577
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.506250
Train Epoch: 67 - Loss: 0.5149, Accuracy: 99.82%

Test set: Average loss: 0.5239, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.516063
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.515971
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.518967
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.515013
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.521613
Train Epoch: 68 - Loss: 0.5147, Accuracy: 99.81%

Test set: Average loss: 0.5239, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.533553
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.509652
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.512451
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.507462
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.518817
Train Epoch: 69 - Loss: 0.5144, Accuracy: 99.82%

Test set: Average loss: 0.5240, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.525800
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.533132
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.508328
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.507446
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.516250
Train Epoch: 70 - Loss: 0.5143, Accuracy: 99.83%

Test set: Average loss: 0.5237, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.516099
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.514666
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.514251
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.523353
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.518867
Train Epoch: 71 - Loss: 0.5142, Accuracy: 99.83%

Test set: Average loss: 0.5237, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.513219
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.528205
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.514662
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.508743
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.521402
Train Epoch: 72 - Loss: 0.5141, Accuracy: 99.84%

Test set: Average loss: 0.5237, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.510854
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.518719
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.517713
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.511073
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.511948
Train Epoch: 73 - Loss: 0.5140, Accuracy: 99.84%

Test set: Average loss: 0.5236, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.516242
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.509737
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.519506
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.520004
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513616
Train Epoch: 74 - Loss: 0.5139, Accuracy: 99.84%

Test set: Average loss: 0.5236, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.522855
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.511823
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.514731
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.536374
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.529146
Train Epoch: 75 - Loss: 0.5139, Accuracy: 99.84%

Test set: Average loss: 0.5236, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.510159
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.515765
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.514922
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.510813
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.514513
Train Epoch: 76 - Loss: 0.5169, Accuracy: 99.75%

Test set: Average loss: 0.5284, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.510255
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.518305
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.509977
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.524670
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.513889
Train Epoch: 77 - Loss: 0.5165, Accuracy: 99.77%

Test set: Average loss: 0.5247, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.525698
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.519558
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.514831
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.526396
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.519933
Train Epoch: 78 - Loss: 0.5159, Accuracy: 99.78%

Test set: Average loss: 0.5247, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.521524
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.522616
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.515835
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.532975
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.517064
Train Epoch: 79 - Loss: 0.5157, Accuracy: 99.77%

Test set: Average loss: 0.5272, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.513620
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.519828
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.521058
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.511296
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.508721
Train Epoch: 80 - Loss: 0.5154, Accuracy: 99.80%

Test set: Average loss: 0.5243, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.516986
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.508434
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.510033
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.514991
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.515986
Train Epoch: 81 - Loss: 0.5150, Accuracy: 99.79%

Test set: Average loss: 0.5258, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.511922
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.514363
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.509256
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.522133
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.515015
Train Epoch: 82 - Loss: 0.5145, Accuracy: 99.80%

Test set: Average loss: 0.5243, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.526931
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.523114
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.508880
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.510701
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.510004
Train Epoch: 83 - Loss: 0.5140, Accuracy: 99.83%

Test set: Average loss: 0.5233, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.513141
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.511814
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.519991
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.512102
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.509829
Train Epoch: 84 - Loss: 0.5138, Accuracy: 99.84%

Test set: Average loss: 0.5233, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508646
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.510803
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.508326
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.512345
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.507169
Train Epoch: 85 - Loss: 0.5134, Accuracy: 99.85%

Test set: Average loss: 0.5229, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.516514
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.520004
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.518050
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.513502
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.512742
Train Epoch: 86 - Loss: 0.5131, Accuracy: 99.85%

Test set: Average loss: 0.5232, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.509736
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.511912
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.507098
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.513080
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.516672
Train Epoch: 87 - Loss: 0.5127, Accuracy: 99.86%

Test set: Average loss: 0.5225, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.510432
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507478
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509547
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.512987
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.507316
Train Epoch: 88 - Loss: 0.5125, Accuracy: 99.86%

Test set: Average loss: 0.5228, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.507955
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.506031
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.511974
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.515187
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.513605
Train Epoch: 89 - Loss: 0.5122, Accuracy: 99.87%

Test set: Average loss: 0.5222, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.513901
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.506659
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.513584
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.516482
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.508431
Train Epoch: 90 - Loss: 0.5120, Accuracy: 99.88%

Test set: Average loss: 0.5220, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.522686
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.510264
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509515
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.507808
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.508611
Train Epoch: 91 - Loss: 0.5118, Accuracy: 99.88%

Test set: Average loss: 0.5224, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.507087
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.512850
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.509932
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.513236
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.506653
Train Epoch: 92 - Loss: 0.5116, Accuracy: 99.89%

Test set: Average loss: 0.5219, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.512532
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.507432
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.507842
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.509269
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.505348
Train Epoch: 93 - Loss: 0.5114, Accuracy: 99.89%

Test set: Average loss: 0.5219, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.509893
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.508055
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.512748
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.524659
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.511526
Train Epoch: 94 - Loss: 0.5113, Accuracy: 99.91%

Test set: Average loss: 0.5219, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.511388
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.516977
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.515669
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.508164
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.507749
Train Epoch: 95 - Loss: 0.5112, Accuracy: 99.90%

Test set: Average loss: 0.5221, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.511765
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.510005
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.508258
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.514786
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.507724
Train Epoch: 96 - Loss: 0.5111, Accuracy: 99.90%

Test set: Average loss: 0.5218, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.506909
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.506975
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.509276
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.513022
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507663
Train Epoch: 97 - Loss: 0.5110, Accuracy: 99.91%

Test set: Average loss: 0.5219, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.507537
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.514244
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.515070
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.510653
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.507639
Train Epoch: 98 - Loss: 0.5110, Accuracy: 99.90%

Test set: Average loss: 0.5218, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.519734
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.509883
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.510960
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.515742
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.512580
Train Epoch: 99 - Loss: 0.5109, Accuracy: 99.91%

Test set: Average loss: 0.5218, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.507114
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.508607
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508892
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.507680
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.507797
Train Epoch: 100 - Loss: 0.5109, Accuracy: 99.91%

Test set: Average loss: 0.5218, Accuracy: 9940/10000 (99.40%)


Training completed. Best Accuracy: 99.42%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.968135
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.835427
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.777797
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.660888
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.657831
Train Epoch: 1 - Loss: 0.8996, Accuracy: 84.95%

Test set: Average loss: 0.6179, Accuracy: 9682/10000 (96.82%)

New best model saved with accuracy: 96.82%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.605845
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.578733
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.635206
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.552240
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.599660
Train Epoch: 2 - Loss: 0.6127, Accuracy: 96.95%

Test set: Average loss: 0.5889, Accuracy: 9771/10000 (97.71%)

New best model saved with accuracy: 97.71%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.602928
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.591091
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.574793
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.604892
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.583904
Train Epoch: 3 - Loss: 0.5844, Accuracy: 97.86%

Test set: Average loss: 0.6089, Accuracy: 9771/10000 (97.71%)

Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.710841
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.573655
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.584743
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.579013
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.547536
Train Epoch: 4 - Loss: 0.5702, Accuracy: 98.34%

Test set: Average loss: 0.5630, Accuracy: 9851/10000 (98.51%)

New best model saved with accuracy: 98.51%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.586792
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.553831
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.554221
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.545046
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.558932
Train Epoch: 5 - Loss: 0.5610, Accuracy: 98.59%

Test set: Average loss: 0.5552, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.564467
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.553080
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.558456
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.561800
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.542062
Train Epoch: 6 - Loss: 0.5548, Accuracy: 98.76%

Test set: Average loss: 0.5622, Accuracy: 9861/10000 (98.61%)

Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.579101
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.545079
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.538723
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.579581
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.551941
Train Epoch: 7 - Loss: 0.5503, Accuracy: 98.85%

Test set: Average loss: 0.5483, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.529378
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.530495
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.554210
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.536767
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.549864
Train Epoch: 8 - Loss: 0.5467, Accuracy: 98.95%

Test set: Average loss: 0.5475, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.531918
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.528971
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.547071
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.552884
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.539076
Train Epoch: 9 - Loss: 0.5438, Accuracy: 99.01%

Test set: Average loss: 0.5474, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.570256
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.546840
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.532310
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.560146
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.523696
Train Epoch: 10 - Loss: 0.5410, Accuracy: 99.12%

Test set: Average loss: 0.5427, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.547671
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.533407
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.535092
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.530358
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.551160
Train Epoch: 11 - Loss: 0.5392, Accuracy: 99.15%

Test set: Average loss: 0.5408, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.527940
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.539840
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.556703
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.555403
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.544424
Train Epoch: 12 - Loss: 0.5373, Accuracy: 99.20%

Test set: Average loss: 0.5424, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.550526
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.523136
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.525699
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.516535
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.531877
Train Epoch: 13 - Loss: 0.5360, Accuracy: 99.21%

Test set: Average loss: 0.5395, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.535853
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.539897
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.528300
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.536456
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.540828
Train Epoch: 14 - Loss: 0.5346, Accuracy: 99.28%

Test set: Average loss: 0.5386, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.556125
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.536439
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.536575
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.526721
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.522346
Train Epoch: 15 - Loss: 0.5336, Accuracy: 99.31%

Test set: Average loss: 0.5369, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.521434
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.518083
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.540757
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.521560
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.527080
Train Epoch: 16 - Loss: 0.5327, Accuracy: 99.34%

Test set: Average loss: 0.5370, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.529513
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.531337
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.527908
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.524975
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.516180
Train Epoch: 17 - Loss: 0.5318, Accuracy: 99.35%

Test set: Average loss: 0.5361, Accuracy: 9906/10000 (99.06%)

New best model saved with accuracy: 99.06%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.557902
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.535536
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.540532
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.513318
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.521027
Train Epoch: 18 - Loss: 0.5312, Accuracy: 99.38%

Test set: Average loss: 0.5360, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.520687
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.534695
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.525596
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.520728
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.539480
Train Epoch: 19 - Loss: 0.5306, Accuracy: 99.41%

Test set: Average loss: 0.5359, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.516747
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.531078
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.518903
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.519375
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.532480
Train Epoch: 20 - Loss: 0.5302, Accuracy: 99.41%

Test set: Average loss: 0.5354, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.521263
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.526516
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.518387
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.517666
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.539881
Train Epoch: 21 - Loss: 0.5298, Accuracy: 99.42%

Test set: Average loss: 0.5362, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.511728
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.527757
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.524411
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.532457
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.533059
Train Epoch: 22 - Loss: 0.5296, Accuracy: 99.41%

Test set: Average loss: 0.5352, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.542778
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.518792
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.531273
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.519874
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.522290
Train Epoch: 23 - Loss: 0.5294, Accuracy: 99.43%

Test set: Average loss: 0.5350, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.543379
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.526407
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.520916
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.521375
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.530434
Train Epoch: 24 - Loss: 0.5292, Accuracy: 99.44%

Test set: Average loss: 0.5350, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.549917
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.519410
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.521980
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.515518
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.551493
Train Epoch: 25 - Loss: 0.5291, Accuracy: 99.43%

Test set: Average loss: 0.5350, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.516682
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.543033
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.536467
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.518749
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.532922
Train Epoch: 26 - Loss: 0.5345, Accuracy: 99.25%

Test set: Average loss: 0.5379, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.534895
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.528804
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.519224
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.526317
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.526991
Train Epoch: 27 - Loss: 0.5329, Accuracy: 99.32%

Test set: Average loss: 0.5374, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.536214
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.531559
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.538250
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.535955
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.520300
Train Epoch: 28 - Loss: 0.5317, Accuracy: 99.35%

Test set: Average loss: 0.5344, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.528487
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.525545
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.523592
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.542322
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.514682
Train Epoch: 29 - Loss: 0.5304, Accuracy: 99.39%

Test set: Average loss: 0.5369, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.532557
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.535756
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.529989
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.552068
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.537792
Train Epoch: 30 - Loss: 0.5294, Accuracy: 99.40%

Test set: Average loss: 0.5331, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.518160
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.525719
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.517697
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.519894
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.519361
Train Epoch: 31 - Loss: 0.5282, Accuracy: 99.44%

Test set: Average loss: 0.5330, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.508657
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.547150
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.528256
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.515491
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.521911
Train Epoch: 32 - Loss: 0.5271, Accuracy: 99.45%

Test set: Average loss: 0.5340, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.517990
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.525449
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.515066
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.529219
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.538913
Train Epoch: 33 - Loss: 0.5260, Accuracy: 99.49%

Test set: Average loss: 0.5315, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.512299
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.544465
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.519458
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.521655
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.515622
Train Epoch: 34 - Loss: 0.5251, Accuracy: 99.53%

Test set: Average loss: 0.5330, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.523642
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.518066
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.518262
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.527213
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.518272
Train Epoch: 35 - Loss: 0.5242, Accuracy: 99.51%

Test set: Average loss: 0.5314, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.516823
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.515683
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.524276
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.518105
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.523617
Train Epoch: 36 - Loss: 0.5236, Accuracy: 99.55%

Test set: Average loss: 0.5323, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.515128
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.520392
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.523019
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.518427
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.528783
Train Epoch: 37 - Loss: 0.5230, Accuracy: 99.57%

Test set: Average loss: 0.5303, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.513795
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.522200
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.524523
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.526614
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.550161
Train Epoch: 38 - Loss: 0.5224, Accuracy: 99.59%

Test set: Average loss: 0.5309, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.513912
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.515563
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.515886
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.521014
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.546921
Train Epoch: 39 - Loss: 0.5217, Accuracy: 99.62%

Test set: Average loss: 0.5291, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.553575
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.527486
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.525507
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.514411
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.511075
Train Epoch: 40 - Loss: 0.5213, Accuracy: 99.63%

Test set: Average loss: 0.5294, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.522385
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.517977
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.515757
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.528613
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.523973
Train Epoch: 41 - Loss: 0.5208, Accuracy: 99.64%

Test set: Average loss: 0.5292, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.513608
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.518589
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.530855
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.517999
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.536822
Train Epoch: 42 - Loss: 0.5204, Accuracy: 99.66%

Test set: Average loss: 0.5295, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.528149
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.518651
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.522103
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.526301
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.509992
Train Epoch: 43 - Loss: 0.5201, Accuracy: 99.65%

Test set: Average loss: 0.5285, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.511559
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.517273
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.516467
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.513054
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.516063
Train Epoch: 44 - Loss: 0.5197, Accuracy: 99.67%

Test set: Average loss: 0.5293, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.520575
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.517105
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.510099
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.512471
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.518378
Train Epoch: 45 - Loss: 0.5195, Accuracy: 99.68%

Test set: Average loss: 0.5282, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.526260
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.528514
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.511859
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.516642
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.516212
Train Epoch: 46 - Loss: 0.5193, Accuracy: 99.68%

Test set: Average loss: 0.5282, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.521484
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.526870
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.516909
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.518127
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.516433
Train Epoch: 47 - Loss: 0.5191, Accuracy: 99.69%

Test set: Average loss: 0.5283, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.511871
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.521411
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.526312
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.517695
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.516868
Train Epoch: 48 - Loss: 0.5190, Accuracy: 99.69%

Test set: Average loss: 0.5282, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.524325
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.523618
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.510956
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.514566
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.516755
Train Epoch: 49 - Loss: 0.5189, Accuracy: 99.70%

Test set: Average loss: 0.5282, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.511320
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.520717
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.528599
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.523825
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.510710
Train Epoch: 50 - Loss: 0.5189, Accuracy: 99.71%

Test set: Average loss: 0.5282, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.511640
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.510182
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.508906
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.512487
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.518086
Train Epoch: 51 - Loss: 0.5229, Accuracy: 99.58%

Test set: Average loss: 0.5298, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.523850
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.514423
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.529912
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.527231
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.514247
Train Epoch: 52 - Loss: 0.5222, Accuracy: 99.58%

Test set: Average loss: 0.5328, Accuracy: 9906/10000 (99.06%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.548143
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.529771
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.513910
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.525098
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.517136
Train Epoch: 53 - Loss: 0.5213, Accuracy: 99.62%

Test set: Average loss: 0.5325, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.522965
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.521704
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.522121
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.522886
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.518574
Train Epoch: 54 - Loss: 0.5210, Accuracy: 99.61%

Test set: Average loss: 0.5303, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.516692
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.525934
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.516724
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.510061
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.537696
Train Epoch: 55 - Loss: 0.5203, Accuracy: 99.66%

Test set: Average loss: 0.5299, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.526712
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.515612
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.525129
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.511650
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.519005
Train Epoch: 56 - Loss: 0.5198, Accuracy: 99.67%

Test set: Average loss: 0.5278, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.516899
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.516128
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.522222
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.508536
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.531491
Train Epoch: 57 - Loss: 0.5190, Accuracy: 99.67%

Test set: Average loss: 0.5295, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.532544
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.516045
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.531129
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.511568
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.511584
Train Epoch: 58 - Loss: 0.5187, Accuracy: 99.68%

Test set: Average loss: 0.5274, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.514522
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.509361
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.521288
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.516153
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.526254
Train Epoch: 59 - Loss: 0.5181, Accuracy: 99.71%

Test set: Average loss: 0.5283, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.510531
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.510549
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.515357
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.510935
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.510906
Train Epoch: 60 - Loss: 0.5175, Accuracy: 99.71%

Test set: Average loss: 0.5290, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.514164
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.508899
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.517906
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.526569
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.512205
Train Epoch: 61 - Loss: 0.5171, Accuracy: 99.74%

Test set: Average loss: 0.5272, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.517943
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.512350
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.530149
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.509159
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.513847
Train Epoch: 62 - Loss: 0.5166, Accuracy: 99.76%

Test set: Average loss: 0.5281, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.514731
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.514583
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.511316
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.510348
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.522211
Train Epoch: 63 - Loss: 0.5163, Accuracy: 99.76%

Test set: Average loss: 0.5259, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.508283
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.512875
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.528154
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.512857
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.507678
Train Epoch: 64 - Loss: 0.5159, Accuracy: 99.78%

Test set: Average loss: 0.5259, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.538930
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.514249
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.509390
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.564292
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.511170
Train Epoch: 65 - Loss: 0.5156, Accuracy: 99.78%

Test set: Average loss: 0.5259, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.525150
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.528113
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.510435
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.510630
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.521344
Train Epoch: 66 - Loss: 0.5152, Accuracy: 99.81%

Test set: Average loss: 0.5275, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.517398
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.514006
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.522456
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.509011
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.523202
Train Epoch: 67 - Loss: 0.5150, Accuracy: 99.81%

Test set: Average loss: 0.5257, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.510634
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.516053
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.511891
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.511652
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.512343
Train Epoch: 68 - Loss: 0.5147, Accuracy: 99.81%

Test set: Average loss: 0.5259, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.510551
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.507880
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.512375
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.510538
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.509215
Train Epoch: 69 - Loss: 0.5146, Accuracy: 99.82%

Test set: Average loss: 0.5259, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.512733
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.512990
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.510551
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.510826
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.514517
Train Epoch: 70 - Loss: 0.5144, Accuracy: 99.83%

Test set: Average loss: 0.5255, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.514809
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.507520
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.508097
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.517893
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.507002
Train Epoch: 71 - Loss: 0.5143, Accuracy: 99.83%

Test set: Average loss: 0.5254, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.511385
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.516062
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.508245
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.528901
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.517796
Train Epoch: 72 - Loss: 0.5141, Accuracy: 99.83%

Test set: Average loss: 0.5252, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.508874
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.516650
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.519666
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.513651
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.524535
Train Epoch: 73 - Loss: 0.5141, Accuracy: 99.84%

Test set: Average loss: 0.5253, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.508945
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.514890
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.510743
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.507992
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.509482
Train Epoch: 74 - Loss: 0.5140, Accuracy: 99.84%

Test set: Average loss: 0.5253, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.512795
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.519620
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.515905
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.509330
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.524396
Train Epoch: 75 - Loss: 0.5140, Accuracy: 99.84%

Test set: Average loss: 0.5253, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.513959
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.518066
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.510370
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.524471
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.514513
Train Epoch: 76 - Loss: 0.5170, Accuracy: 99.74%

Test set: Average loss: 0.5283, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.511073
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.517321
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.537140
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.513069
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.511780
Train Epoch: 77 - Loss: 0.5168, Accuracy: 99.73%

Test set: Average loss: 0.5274, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.531673
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.522596
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.512082
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.516849
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.512215
Train Epoch: 78 - Loss: 0.5160, Accuracy: 99.76%

Test set: Average loss: 0.5308, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.511055
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.528719
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.513931
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.511507
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.513780
Train Epoch: 79 - Loss: 0.5159, Accuracy: 99.78%

Test set: Average loss: 0.5279, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.527514
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.518186
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.507554
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.510108
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.517698
Train Epoch: 80 - Loss: 0.5155, Accuracy: 99.78%

Test set: Average loss: 0.5262, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.513596
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.511885
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.507912
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.525254
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.509665
Train Epoch: 81 - Loss: 0.5153, Accuracy: 99.77%

Test set: Average loss: 0.5251, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.514955
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.516337
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.511691
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.516092
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.520343
Train Epoch: 82 - Loss: 0.5146, Accuracy: 99.81%

Test set: Average loss: 0.5255, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.512053
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.508970
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.510910
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.517598
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.519846
Train Epoch: 83 - Loss: 0.5144, Accuracy: 99.79%

Test set: Average loss: 0.5258, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.519748
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.509866
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.515321
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.518287
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.516489
Train Epoch: 84 - Loss: 0.5138, Accuracy: 99.83%

Test set: Average loss: 0.5261, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508926
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.517839
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.515495
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.516510
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.517929
Train Epoch: 85 - Loss: 0.5136, Accuracy: 99.83%

Test set: Average loss: 0.5260, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.514611
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.522204
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.542573
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.512949
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.508614
Train Epoch: 86 - Loss: 0.5132, Accuracy: 99.84%

Test set: Average loss: 0.5266, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.524049
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.511742
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.516495
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.508286
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.516310
Train Epoch: 87 - Loss: 0.5129, Accuracy: 99.83%

Test set: Average loss: 0.5250, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.507449
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.514384
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509055
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.519157
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.508343
Train Epoch: 88 - Loss: 0.5127, Accuracy: 99.86%

Test set: Average loss: 0.5239, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.507923
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.513422
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.513178
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.516460
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.508373
Train Epoch: 89 - Loss: 0.5124, Accuracy: 99.87%

Test set: Average loss: 0.5239, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.507476
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.509160
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.510280
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508626
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.511916
Train Epoch: 90 - Loss: 0.5121, Accuracy: 99.88%

Test set: Average loss: 0.5240, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.512082
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.517351
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.507249
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.508985
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.529728
Train Epoch: 91 - Loss: 0.5119, Accuracy: 99.88%

Test set: Average loss: 0.5236, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.515163
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510285
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.507924
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.509115
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.508986
Train Epoch: 92 - Loss: 0.5117, Accuracy: 99.89%

Test set: Average loss: 0.5245, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.513778
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.507471
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.520486
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.512457
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.507054
Train Epoch: 93 - Loss: 0.5116, Accuracy: 99.89%

Test set: Average loss: 0.5236, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.520045
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.506949
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.510649
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.513528
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.510199
Train Epoch: 94 - Loss: 0.5114, Accuracy: 99.89%

Test set: Average loss: 0.5236, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.506521
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.512178
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506871
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.508016
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509869
Train Epoch: 95 - Loss: 0.5113, Accuracy: 99.90%

Test set: Average loss: 0.5242, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.525179
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.511502
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.508187
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508923
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.508624
Train Epoch: 96 - Loss: 0.5112, Accuracy: 99.90%

Test set: Average loss: 0.5237, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.516936
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509950
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.530385
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.512767
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.506072
Train Epoch: 97 - Loss: 0.5111, Accuracy: 99.90%

Test set: Average loss: 0.5236, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.507356
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.508052
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.513368
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.507141
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.508788
Train Epoch: 98 - Loss: 0.5111, Accuracy: 99.91%

Test set: Average loss: 0.5235, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.507630
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.513674
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.505295
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.518175
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.508156
Train Epoch: 99 - Loss: 0.5110, Accuracy: 99.91%

Test set: Average loss: 0.5236, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.506181
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.508695
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508809
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.507301
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.526866
Train Epoch: 100 - Loss: 0.5110, Accuracy: 99.91%

Test set: Average loss: 0.5236, Accuracy: 9923/10000 (99.23%)


Training completed. Best Accuracy: 99.27%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.694262
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.847530
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.725802
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.725078
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.632821
Train Epoch: 1 - Loss: 0.9405, Accuracy: 83.08%

Test set: Average loss: 0.6475, Accuracy: 9591/10000 (95.91%)

New best model saved with accuracy: 95.91%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.642005
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.603419
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.619158
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.617943
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.604442
Train Epoch: 2 - Loss: 0.6231, Accuracy: 96.59%

Test set: Average loss: 0.6081, Accuracy: 9747/10000 (97.47%)

New best model saved with accuracy: 97.47%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.667976
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.581271
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.613423
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.618223
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.613832
Train Epoch: 3 - Loss: 0.5904, Accuracy: 97.70%

Test set: Average loss: 0.5786, Accuracy: 9796/10000 (97.96%)

New best model saved with accuracy: 97.96%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.587287
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.562549
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.562554
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.591459
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.585291
Train Epoch: 4 - Loss: 0.5748, Accuracy: 98.19%

Test set: Average loss: 0.5775, Accuracy: 9792/10000 (97.92%)

Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.567237
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.583996
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.553998
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.566873
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.542192
Train Epoch: 5 - Loss: 0.5646, Accuracy: 98.45%

Test set: Average loss: 0.5594, Accuracy: 9845/10000 (98.45%)

New best model saved with accuracy: 98.45%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.587289
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.584936
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.553431
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.529558
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.545020
Train Epoch: 6 - Loss: 0.5578, Accuracy: 98.68%

Test set: Average loss: 0.5593, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.558619
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.549624
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.559650
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.530712
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.537918
Train Epoch: 7 - Loss: 0.5528, Accuracy: 98.80%

Test set: Average loss: 0.5489, Accuracy: 9873/10000 (98.73%)

New best model saved with accuracy: 98.73%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.560242
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.526691
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.531558
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.546823
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.556584
Train Epoch: 8 - Loss: 0.5484, Accuracy: 98.89%

Test set: Average loss: 0.5466, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.538549
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.563386
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.533114
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.552479
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.534119
Train Epoch: 9 - Loss: 0.5453, Accuracy: 99.02%

Test set: Average loss: 0.5449, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.571910
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.552873
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.557425
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.533995
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.523811
Train Epoch: 10 - Loss: 0.5426, Accuracy: 99.10%

Test set: Average loss: 0.5423, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.556899
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.543770
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.568498
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.536074
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.522443
Train Epoch: 11 - Loss: 0.5406, Accuracy: 99.13%

Test set: Average loss: 0.5415, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.528779
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.525401
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.527527
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.538109
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.548647
Train Epoch: 12 - Loss: 0.5387, Accuracy: 99.18%

Test set: Average loss: 0.5411, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.529323
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.539557
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.514593
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.553233
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.535106
Train Epoch: 13 - Loss: 0.5369, Accuracy: 99.25%

Test set: Average loss: 0.5397, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.539402
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.545828
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.546884
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.527593
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.532693
Train Epoch: 14 - Loss: 0.5357, Accuracy: 99.28%

Test set: Average loss: 0.5379, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.529506
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.529434
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.537896
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.531068
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.544592
Train Epoch: 15 - Loss: 0.5345, Accuracy: 99.30%

Test set: Average loss: 0.5378, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.527325
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.533817
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.520008
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.525256
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.519692
Train Epoch: 16 - Loss: 0.5334, Accuracy: 99.34%

Test set: Average loss: 0.5361, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.523657
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.526587
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.532424
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.524545
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.524583
Train Epoch: 17 - Loss: 0.5326, Accuracy: 99.35%

Test set: Average loss: 0.5353, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.520213
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.536569
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.554174
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.526362
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.526986
Train Epoch: 18 - Loss: 0.5319, Accuracy: 99.40%

Test set: Average loss: 0.5351, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.570575
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.518701
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.516715
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.536474
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.545055
Train Epoch: 19 - Loss: 0.5313, Accuracy: 99.42%

Test set: Average loss: 0.5348, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.533826
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.519963
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.521569
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.526138
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.521849
Train Epoch: 20 - Loss: 0.5308, Accuracy: 99.42%

Test set: Average loss: 0.5344, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.540943
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.527183
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.557454
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.517228
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.530310
Train Epoch: 21 - Loss: 0.5305, Accuracy: 99.44%

Test set: Average loss: 0.5345, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.530354
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.533634
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.542615
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.535471
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.531250
Train Epoch: 22 - Loss: 0.5301, Accuracy: 99.45%

Test set: Average loss: 0.5342, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.529608
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.517633
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.523181
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.514614
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.524747
Train Epoch: 23 - Loss: 0.5299, Accuracy: 99.45%

Test set: Average loss: 0.5342, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.559259
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.521076
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.542524
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.527436
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.520123
Train Epoch: 24 - Loss: 0.5298, Accuracy: 99.47%

Test set: Average loss: 0.5342, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.519940
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.514995
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.532100
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.522550
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.514164
Train Epoch: 25 - Loss: 0.5297, Accuracy: 99.47%

Test set: Average loss: 0.5342, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.520807
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.529333
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.532235
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.541708
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.524719
Train Epoch: 26 - Loss: 0.5357, Accuracy: 99.25%

Test set: Average loss: 0.5413, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.518302
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.553999
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.567249
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.532389
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.528417
Train Epoch: 27 - Loss: 0.5339, Accuracy: 99.31%

Test set: Average loss: 0.5359, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.546106
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.524726
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.534918
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.529208
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.527359
Train Epoch: 28 - Loss: 0.5324, Accuracy: 99.34%

Test set: Average loss: 0.5349, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.522582
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.524540
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.526021
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.528822
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.515405
Train Epoch: 29 - Loss: 0.5311, Accuracy: 99.37%

Test set: Average loss: 0.5350, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.519406
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.520354
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.517767
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.534307
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.521315
Train Epoch: 30 - Loss: 0.5297, Accuracy: 99.42%

Test set: Average loss: 0.5316, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.519001
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.521381
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.529967
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.514097
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.522290
Train Epoch: 31 - Loss: 0.5284, Accuracy: 99.46%

Test set: Average loss: 0.5348, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.523241
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.523985
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.518436
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.536720
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.538999
Train Epoch: 32 - Loss: 0.5274, Accuracy: 99.49%

Test set: Average loss: 0.5325, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.521072
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.530286
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.533589
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.542858
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.553745
Train Epoch: 33 - Loss: 0.5265, Accuracy: 99.52%

Test set: Average loss: 0.5304, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.523591
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.511732
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.532443
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.546992
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.536445
Train Epoch: 34 - Loss: 0.5255, Accuracy: 99.52%

Test set: Average loss: 0.5307, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.537301
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.512622
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.516162
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.527322
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.518820
Train Epoch: 35 - Loss: 0.5247, Accuracy: 99.58%

Test set: Average loss: 0.5288, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.522571
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.516310
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.527929
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.518561
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.531067
Train Epoch: 36 - Loss: 0.5240, Accuracy: 99.56%

Test set: Average loss: 0.5286, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.534699
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.515386
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.517712
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.518818
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.518539
Train Epoch: 37 - Loss: 0.5232, Accuracy: 99.59%

Test set: Average loss: 0.5282, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.512603
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.524919
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.511823
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.531797
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.532901
Train Epoch: 38 - Loss: 0.5226, Accuracy: 99.62%

Test set: Average loss: 0.5281, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.521943
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.518104
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.524352
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.518286
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.522025
Train Epoch: 39 - Loss: 0.5220, Accuracy: 99.63%

Test set: Average loss: 0.5275, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.519017
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.534991
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.516066
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.511078
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.525349
Train Epoch: 40 - Loss: 0.5215, Accuracy: 99.65%

Test set: Average loss: 0.5271, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.519328
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.519737
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.515123
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.523919
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.521686
Train Epoch: 41 - Loss: 0.5210, Accuracy: 99.67%

Test set: Average loss: 0.5272, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.538508
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.519783
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.510666
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.533723
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.514739
Train Epoch: 42 - Loss: 0.5206, Accuracy: 99.67%

Test set: Average loss: 0.5269, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.521035
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.522827
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.514985
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.513660
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.516737
Train Epoch: 43 - Loss: 0.5202, Accuracy: 99.71%

Test set: Average loss: 0.5268, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.525196
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.513698
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.513394
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.518224
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.519579
Train Epoch: 44 - Loss: 0.5200, Accuracy: 99.69%

Test set: Average loss: 0.5264, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.522885
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.510499
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.510657
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.510401
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.518371
Train Epoch: 45 - Loss: 0.5197, Accuracy: 99.70%

Test set: Average loss: 0.5263, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.515110
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.513911
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.520396
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.516582
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.527432
Train Epoch: 46 - Loss: 0.5195, Accuracy: 99.72%

Test set: Average loss: 0.5263, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.515940
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.524979
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.509969
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.512697
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.520187
Train Epoch: 47 - Loss: 0.5194, Accuracy: 99.72%

Test set: Average loss: 0.5263, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.514356
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.514415
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.516940
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.508467
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.512925
Train Epoch: 48 - Loss: 0.5192, Accuracy: 99.72%

Test set: Average loss: 0.5261, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.514088
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.510917
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.523546
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.517207
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.510240
Train Epoch: 49 - Loss: 0.5192, Accuracy: 99.73%

Test set: Average loss: 0.5261, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.530854
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.514962
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.518905
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.516773
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.524762
Train Epoch: 50 - Loss: 0.5191, Accuracy: 99.73%

Test set: Average loss: 0.5261, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.535297
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.530062
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.512598
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.522812
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.515513
Train Epoch: 51 - Loss: 0.5231, Accuracy: 99.60%

Test set: Average loss: 0.5307, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.539138
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.518905
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.535942
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.518597
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.520259
Train Epoch: 52 - Loss: 0.5226, Accuracy: 99.62%

Test set: Average loss: 0.5282, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.513786
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.526832
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.518397
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.519442
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.513000
Train Epoch: 53 - Loss: 0.5219, Accuracy: 99.62%

Test set: Average loss: 0.5285, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.541957
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.521175
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.510995
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.519107
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.514966
Train Epoch: 54 - Loss: 0.5213, Accuracy: 99.66%

Test set: Average loss: 0.5274, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.513746
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.511223
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.522599
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.514853
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.554143
Train Epoch: 55 - Loss: 0.5205, Accuracy: 99.66%

Test set: Average loss: 0.5279, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.524214
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.519424
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.518382
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.520964
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.512523
Train Epoch: 56 - Loss: 0.5202, Accuracy: 99.66%

Test set: Average loss: 0.5264, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.512777
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.521988
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.512261
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.527332
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.511548
Train Epoch: 57 - Loss: 0.5194, Accuracy: 99.70%

Test set: Average loss: 0.5252, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.517719
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.512411
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.514632
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.528419
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.514262
Train Epoch: 58 - Loss: 0.5190, Accuracy: 99.71%

Test set: Average loss: 0.5252, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.520854
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.517489
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.515323
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.509183
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.521668
Train Epoch: 59 - Loss: 0.5183, Accuracy: 99.72%

Test set: Average loss: 0.5253, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.553539
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.513397
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.516544
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.509065
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.512004
Train Epoch: 60 - Loss: 0.5178, Accuracy: 99.74%

Test set: Average loss: 0.5266, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.513924
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.535291
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.510218
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.526125
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.518187
Train Epoch: 61 - Loss: 0.5174, Accuracy: 99.75%

Test set: Average loss: 0.5246, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.517287
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.508191
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.515884
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.509493
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.526789
Train Epoch: 62 - Loss: 0.5170, Accuracy: 99.76%

Test set: Average loss: 0.5245, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.509752
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.512643
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.510398
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.515955
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.521115
Train Epoch: 63 - Loss: 0.5165, Accuracy: 99.78%

Test set: Average loss: 0.5237, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.518820
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.525182
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.513098
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.520555
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.514784
Train Epoch: 64 - Loss: 0.5162, Accuracy: 99.79%

Test set: Average loss: 0.5240, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.513391
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.512768
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.513084
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.518150
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.525543
Train Epoch: 65 - Loss: 0.5159, Accuracy: 99.79%

Test set: Average loss: 0.5238, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.514875
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.510913
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.526814
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.519304
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.511220
Train Epoch: 66 - Loss: 0.5155, Accuracy: 99.80%

Test set: Average loss: 0.5239, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.517726
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.510780
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.519403
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.515148
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.518834
Train Epoch: 67 - Loss: 0.5153, Accuracy: 99.81%

Test set: Average loss: 0.5234, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.540659
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.510089
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.511479
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.515893
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.514856
Train Epoch: 68 - Loss: 0.5150, Accuracy: 99.81%

Test set: Average loss: 0.5235, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.512745
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.526389
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.525084
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.516383
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.511621
Train Epoch: 69 - Loss: 0.5148, Accuracy: 99.83%

Test set: Average loss: 0.5234, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.507235
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.510061
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.511349
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.510674
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.508101
Train Epoch: 70 - Loss: 0.5147, Accuracy: 99.84%

Test set: Average loss: 0.5232, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.509379
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.511338
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.512458
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.510720
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.516269
Train Epoch: 71 - Loss: 0.5145, Accuracy: 99.84%

Test set: Average loss: 0.5230, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.511869
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.515598
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509103
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.514794
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.507812
Train Epoch: 72 - Loss: 0.5144, Accuracy: 99.84%

Test set: Average loss: 0.5230, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.510491
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508103
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.517169
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.509305
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.515128
Train Epoch: 73 - Loss: 0.5143, Accuracy: 99.85%

Test set: Average loss: 0.5230, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.511694
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.532184
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.519317
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.516671
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.511353
Train Epoch: 74 - Loss: 0.5143, Accuracy: 99.85%

Test set: Average loss: 0.5230, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.518474
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.509097
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.508073
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.515892
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.512632
Train Epoch: 75 - Loss: 0.5142, Accuracy: 99.85%

Test set: Average loss: 0.5230, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.512797
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.511789
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.516038
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.516101
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.510613
Train Epoch: 76 - Loss: 0.5172, Accuracy: 99.73%

Test set: Average loss: 0.5244, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.522663
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.511101
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.513054
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.513552
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.514457
Train Epoch: 77 - Loss: 0.5171, Accuracy: 99.75%

Test set: Average loss: 0.5238, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.522022
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.522236
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.521335
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.517002
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.518725
Train Epoch: 78 - Loss: 0.5167, Accuracy: 99.76%

Test set: Average loss: 0.5260, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.535982
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.513135
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.518099
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.513221
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.521030
Train Epoch: 79 - Loss: 0.5161, Accuracy: 99.78%

Test set: Average loss: 0.5336, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.546110
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.516809
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.510048
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.512013
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.517178
Train Epoch: 80 - Loss: 0.5161, Accuracy: 99.77%

Test set: Average loss: 0.5228, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.509974
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.516148
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.512503
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.514256
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.528051
Train Epoch: 81 - Loss: 0.5153, Accuracy: 99.78%

Test set: Average loss: 0.5241, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.517112
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.518834
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.514483
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.522188
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.508899
Train Epoch: 82 - Loss: 0.5151, Accuracy: 99.80%

Test set: Average loss: 0.5238, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.507971
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512805
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.515062
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.509478
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.511505
Train Epoch: 83 - Loss: 0.5147, Accuracy: 99.80%

Test set: Average loss: 0.5228, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.507058
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.513455
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.520528
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.510426
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.508711
Train Epoch: 84 - Loss: 0.5143, Accuracy: 99.82%

Test set: Average loss: 0.5234, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.519306
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.508160
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.511582
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.514698
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.509492
Train Epoch: 85 - Loss: 0.5139, Accuracy: 99.84%

Test set: Average loss: 0.5228, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.507624
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.512937
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.517802
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.513776
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.518214
Train Epoch: 86 - Loss: 0.5135, Accuracy: 99.83%

Test set: Average loss: 0.5221, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.507697
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.506886
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.512420
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.529849
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.508157
Train Epoch: 87 - Loss: 0.5133, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.510989
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.513931
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509349
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.519792
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.511516
Train Epoch: 88 - Loss: 0.5129, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.514145
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.511684
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.512219
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.512710
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.512640
Train Epoch: 89 - Loss: 0.5126, Accuracy: 99.86%

Test set: Average loss: 0.5223, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.523185
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513198
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.507981
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508321
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.514861
Train Epoch: 90 - Loss: 0.5125, Accuracy: 99.87%

Test set: Average loss: 0.5217, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.507729
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.510711
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.514396
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.518300
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.508624
Train Epoch: 91 - Loss: 0.5122, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.505429
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.534028
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510639
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.511065
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.508348
Train Epoch: 92 - Loss: 0.5120, Accuracy: 99.89%

Test set: Average loss: 0.5214, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.513457
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.508054
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.508796
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.517581
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.510555
Train Epoch: 93 - Loss: 0.5118, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.507188
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509689
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.516307
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.508576
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.515519
Train Epoch: 94 - Loss: 0.5117, Accuracy: 99.89%

Test set: Average loss: 0.5213, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.510732
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.510130
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.516417
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.505314
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.511959
Train Epoch: 95 - Loss: 0.5115, Accuracy: 99.89%

Test set: Average loss: 0.5213, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.510133
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.515581
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507963
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.519047
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.507670
Train Epoch: 96 - Loss: 0.5115, Accuracy: 99.89%

Test set: Average loss: 0.5213, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.511862
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.523302
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.516342
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.509149
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.512730
Train Epoch: 97 - Loss: 0.5114, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.509099
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.509236
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.506880
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.509189
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.517870
Train Epoch: 98 - Loss: 0.5113, Accuracy: 99.90%

Test set: Average loss: 0.5212, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.506199
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.507993
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.510594
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.506514
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.507997
Train Epoch: 99 - Loss: 0.5113, Accuracy: 99.90%

Test set: Average loss: 0.5211, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.508018
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.511389
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.510189
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.513116
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.525139
Train Epoch: 100 - Loss: 0.5113, Accuracy: 99.90%

Test set: Average loss: 0.5211, Accuracy: 9936/10000 (99.36%)


Training completed. Best Accuracy: 99.41%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.431994
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.926112
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.728941
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.689063
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.697196
Train Epoch: 1 - Loss: 0.9275, Accuracy: 83.61%

Test set: Average loss: 0.6657, Accuracy: 9568/10000 (95.68%)

New best model saved with accuracy: 95.68%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.677584
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.711410
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.595145
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.581748
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.587378
Train Epoch: 2 - Loss: 0.6122, Accuracy: 97.13%

Test set: Average loss: 0.5912, Accuracy: 9766/10000 (97.66%)

New best model saved with accuracy: 97.66%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.579492
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.582631
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.582285
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.593513
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.545420
Train Epoch: 3 - Loss: 0.5838, Accuracy: 97.97%

Test set: Average loss: 0.5706, Accuracy: 9851/10000 (98.51%)

New best model saved with accuracy: 98.51%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.566336
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.550651
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.554202
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.574908
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.595252
Train Epoch: 4 - Loss: 0.5701, Accuracy: 98.31%

Test set: Average loss: 0.5641, Accuracy: 9852/10000 (98.52%)

New best model saved with accuracy: 98.52%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.568850
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.555471
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.529894
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.554879
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.530371
Train Epoch: 5 - Loss: 0.5611, Accuracy: 98.55%

Test set: Average loss: 0.5533, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.563021
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.540410
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.548152
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.583189
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.544859
Train Epoch: 6 - Loss: 0.5549, Accuracy: 98.73%

Test set: Average loss: 0.5541, Accuracy: 9876/10000 (98.76%)

Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.570223
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.552083
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.548371
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.574731
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.543806
Train Epoch: 7 - Loss: 0.5501, Accuracy: 98.87%

Test set: Average loss: 0.5534, Accuracy: 9880/10000 (98.80%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.579935
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.531529
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.549223
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.552864
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.535999
Train Epoch: 8 - Loss: 0.5462, Accuracy: 98.96%

Test set: Average loss: 0.5453, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.564523
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.552967
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.531179
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.522638
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.545663
Train Epoch: 9 - Loss: 0.5432, Accuracy: 99.06%

Test set: Average loss: 0.5439, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.529466
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.556256
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.546594
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.535605
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.544689
Train Epoch: 10 - Loss: 0.5408, Accuracy: 99.11%

Test set: Average loss: 0.5417, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.546714
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.529850
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.523257
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.533337
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.526704
Train Epoch: 11 - Loss: 0.5387, Accuracy: 99.18%

Test set: Average loss: 0.5403, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.546884
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.542872
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.574438
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.525784
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.536467
Train Epoch: 12 - Loss: 0.5366, Accuracy: 99.25%

Test set: Average loss: 0.5385, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.535661
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.531383
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.526985
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.543485
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.526663
Train Epoch: 13 - Loss: 0.5352, Accuracy: 99.28%

Test set: Average loss: 0.5404, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.545006
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.517112
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.534453
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.543912
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.521763
Train Epoch: 14 - Loss: 0.5339, Accuracy: 99.34%

Test set: Average loss: 0.5381, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.536763
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.539937
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.546449
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.519779
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.517845
Train Epoch: 15 - Loss: 0.5328, Accuracy: 99.36%

Test set: Average loss: 0.5359, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.547561
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.540454
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.537768
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.557537
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.515910
Train Epoch: 16 - Loss: 0.5320, Accuracy: 99.38%

Test set: Average loss: 0.5356, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.551274
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.521778
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.530024
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.527118
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.533101
Train Epoch: 17 - Loss: 0.5311, Accuracy: 99.42%

Test set: Average loss: 0.5353, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.574471
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.519884
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.523027
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.519262
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.534070
Train Epoch: 18 - Loss: 0.5305, Accuracy: 99.43%

Test set: Average loss: 0.5348, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.539484
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.526372
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.535016
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.542254
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.517401
Train Epoch: 19 - Loss: 0.5299, Accuracy: 99.44%

Test set: Average loss: 0.5345, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.512387
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.539492
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.532287
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.562843
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.533079
Train Epoch: 20 - Loss: 0.5295, Accuracy: 99.45%

Test set: Average loss: 0.5342, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.526219
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.528888
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.527540
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.519390
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.539809
Train Epoch: 21 - Loss: 0.5291, Accuracy: 99.45%

Test set: Average loss: 0.5340, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.525116
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.524627
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.525897
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.513494
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.521694
Train Epoch: 22 - Loss: 0.5288, Accuracy: 99.47%

Test set: Average loss: 0.5338, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.520227
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.520262
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.536383
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.519010
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.525245
Train Epoch: 23 - Loss: 0.5286, Accuracy: 99.47%

Test set: Average loss: 0.5338, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.523721
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.547256
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.532829
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.517047
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.523121
Train Epoch: 24 - Loss: 0.5285, Accuracy: 99.47%

Test set: Average loss: 0.5337, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.542551
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.522092
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.536758
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.522194
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.524347
Train Epoch: 25 - Loss: 0.5284, Accuracy: 99.48%

Test set: Average loss: 0.5336, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.538793
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.525272
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.541685
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.536413
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.558518
Train Epoch: 26 - Loss: 0.5338, Accuracy: 99.32%

Test set: Average loss: 0.5355, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.552059
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.546224
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.521765
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.515258
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.528690
Train Epoch: 27 - Loss: 0.5320, Accuracy: 99.35%

Test set: Average loss: 0.5339, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.516546
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.542306
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.516497
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.526251
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.522168
Train Epoch: 28 - Loss: 0.5306, Accuracy: 99.40%

Test set: Average loss: 0.5335, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.538283
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.539444
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.529222
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.532458
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.526466
Train Epoch: 29 - Loss: 0.5294, Accuracy: 99.42%

Test set: Average loss: 0.5322, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.511846
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.529969
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.519616
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.535447
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.530943
Train Epoch: 30 - Loss: 0.5281, Accuracy: 99.47%

Test set: Average loss: 0.5351, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.521576
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.553896
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.517092
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.547906
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.510871
Train Epoch: 31 - Loss: 0.5269, Accuracy: 99.50%

Test set: Average loss: 0.5301, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.520567
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.533242
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.525356
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.525154
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.517862
Train Epoch: 32 - Loss: 0.5260, Accuracy: 99.52%

Test set: Average loss: 0.5310, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.512827
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.525373
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.537941
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.523854
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.515072
Train Epoch: 33 - Loss: 0.5251, Accuracy: 99.54%

Test set: Average loss: 0.5290, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.516680
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.518641
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.548927
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.519435
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.517874
Train Epoch: 34 - Loss: 0.5241, Accuracy: 99.57%

Test set: Average loss: 0.5297, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.516803
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.533638
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.528689
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.547910
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.524142
Train Epoch: 35 - Loss: 0.5233, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.515144
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.514454
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.517097
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.517028
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.522581
Train Epoch: 36 - Loss: 0.5227, Accuracy: 99.58%

Test set: Average loss: 0.5283, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.514876
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.539340
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.512186
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.518936
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.530859
Train Epoch: 37 - Loss: 0.5220, Accuracy: 99.63%

Test set: Average loss: 0.5283, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.519121
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.518118
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.535061
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.512889
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.510139
Train Epoch: 38 - Loss: 0.5213, Accuracy: 99.64%

Test set: Average loss: 0.5280, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.518613
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.521147
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.517854
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.522105
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.515759
Train Epoch: 39 - Loss: 0.5208, Accuracy: 99.65%

Test set: Average loss: 0.5270, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.529316
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.523924
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.510626
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.510359
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.518343
Train Epoch: 40 - Loss: 0.5203, Accuracy: 99.66%

Test set: Average loss: 0.5271, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.513628
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.512840
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.518530
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.509608
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.540674
Train Epoch: 41 - Loss: 0.5199, Accuracy: 99.66%

Test set: Average loss: 0.5275, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.517289
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.518583
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.518325
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.512552
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.519306
Train Epoch: 42 - Loss: 0.5196, Accuracy: 99.70%

Test set: Average loss: 0.5265, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.515606
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.519533
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.515155
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.525925
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.511607
Train Epoch: 43 - Loss: 0.5192, Accuracy: 99.69%

Test set: Average loss: 0.5266, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.515444
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.525043
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.530096
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.515059
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.513613
Train Epoch: 44 - Loss: 0.5189, Accuracy: 99.70%

Test set: Average loss: 0.5264, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.516037
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.522691
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.516058
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.523827
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.512609
Train Epoch: 45 - Loss: 0.5188, Accuracy: 99.71%

Test set: Average loss: 0.5263, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.524601
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.531876
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.580369
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.521248
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.521752
Train Epoch: 46 - Loss: 0.5185, Accuracy: 99.72%

Test set: Average loss: 0.5262, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.523063
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.525005
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.510253
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.508764
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.522889
Train Epoch: 47 - Loss: 0.5184, Accuracy: 99.73%

Test set: Average loss: 0.5261, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.514603
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.522412
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.521251
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.511008
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.519362
Train Epoch: 48 - Loss: 0.5183, Accuracy: 99.74%

Test set: Average loss: 0.5261, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.528564
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.514639
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.511480
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.532377
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.509870
Train Epoch: 49 - Loss: 0.5182, Accuracy: 99.74%

Test set: Average loss: 0.5261, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.513254
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.518539
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.514398
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.512141
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.540862
Train Epoch: 50 - Loss: 0.5182, Accuracy: 99.74%

Test set: Average loss: 0.5261, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.508522
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.517991
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.517066
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.520888
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.532082
Train Epoch: 51 - Loss: 0.5218, Accuracy: 99.62%

Test set: Average loss: 0.5275, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.520432
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.544402
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.528353
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.522934
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.521661
Train Epoch: 52 - Loss: 0.5212, Accuracy: 99.64%

Test set: Average loss: 0.5268, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.520721
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.514816
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.534342
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.509824
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.520790
Train Epoch: 53 - Loss: 0.5207, Accuracy: 99.62%

Test set: Average loss: 0.5273, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.516436
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.512246
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.520521
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.508451
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.524000
Train Epoch: 54 - Loss: 0.5204, Accuracy: 99.64%

Test set: Average loss: 0.5283, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.522959
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.514038
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.520403
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.518120
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.521259
Train Epoch: 55 - Loss: 0.5196, Accuracy: 99.66%

Test set: Average loss: 0.5318, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.518205
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.529579
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.508119
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.516335
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.528684
Train Epoch: 56 - Loss: 0.5188, Accuracy: 99.70%

Test set: Average loss: 0.5272, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.516953
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.549933
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.512169
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.518863
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.532348
Train Epoch: 57 - Loss: 0.5183, Accuracy: 99.70%

Test set: Average loss: 0.5263, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.532171
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.512303
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.517387
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.522453
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.520878
Train Epoch: 58 - Loss: 0.5179, Accuracy: 99.71%

Test set: Average loss: 0.5265, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.509453
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.520242
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.514695
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.514063
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.510438
Train Epoch: 59 - Loss: 0.5173, Accuracy: 99.75%

Test set: Average loss: 0.5267, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.517439
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.522793
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.543033
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.535161
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.511672
Train Epoch: 60 - Loss: 0.5168, Accuracy: 99.75%

Test set: Average loss: 0.5245, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.511384
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.511054
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.512044
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.524707
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.521088
Train Epoch: 61 - Loss: 0.5163, Accuracy: 99.77%

Test set: Average loss: 0.5244, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.515856
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.516510
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.518587
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.510556
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.511840
Train Epoch: 62 - Loss: 0.5159, Accuracy: 99.78%

Test set: Average loss: 0.5237, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.509011
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.513804
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.507887
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.519797
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.510045
Train Epoch: 63 - Loss: 0.5156, Accuracy: 99.78%

Test set: Average loss: 0.5238, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.513525
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.508247
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.512975
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.508489
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.510693
Train Epoch: 64 - Loss: 0.5152, Accuracy: 99.81%

Test set: Average loss: 0.5244, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.515361
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.511540
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.512522
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.514211
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.521410
Train Epoch: 65 - Loss: 0.5148, Accuracy: 99.81%

Test set: Average loss: 0.5236, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.511205
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.510819
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.519633
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.506836
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.514673
Train Epoch: 66 - Loss: 0.5146, Accuracy: 99.82%

Test set: Average loss: 0.5234, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.510660
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.509846
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.515988
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.509976
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.515050
Train Epoch: 67 - Loss: 0.5143, Accuracy: 99.83%

Test set: Average loss: 0.5240, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.509494
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.513784
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.513189
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.509840
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.509563
Train Epoch: 68 - Loss: 0.5141, Accuracy: 99.83%

Test set: Average loss: 0.5236, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.517545
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.511145
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.526199
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.505649
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.510619
Train Epoch: 69 - Loss: 0.5140, Accuracy: 99.83%

Test set: Average loss: 0.5231, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.523916
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.508674
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.509516
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.509280
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.509089
Train Epoch: 70 - Loss: 0.5138, Accuracy: 99.83%

Test set: Average loss: 0.5232, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.508037
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.514924
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.512795
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.516189
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.511262
Train Epoch: 71 - Loss: 0.5136, Accuracy: 99.85%

Test set: Average loss: 0.5233, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.519443
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.521242
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.518436
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.523209
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.511771
Train Epoch: 72 - Loss: 0.5136, Accuracy: 99.85%

Test set: Average loss: 0.5232, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.510483
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.522920
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.510864
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.511481
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.506151
Train Epoch: 73 - Loss: 0.5135, Accuracy: 99.84%

Test set: Average loss: 0.5231, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.511378
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.508939
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.512011
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.514359
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.515704
Train Epoch: 74 - Loss: 0.5134, Accuracy: 99.84%

Test set: Average loss: 0.5231, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.510646
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.511174
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.513388
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.509857
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.509360
Train Epoch: 75 - Loss: 0.5134, Accuracy: 99.84%

Test set: Average loss: 0.5231, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.511200
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.510192
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.509674
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.524635
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.513925
Train Epoch: 76 - Loss: 0.5163, Accuracy: 99.77%

Test set: Average loss: 0.5233, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.522431
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.514803
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.509146
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.517154
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.511359
Train Epoch: 77 - Loss: 0.5158, Accuracy: 99.78%

Test set: Average loss: 0.5238, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.513682
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.512953
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.510823
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.516327
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.516742
Train Epoch: 78 - Loss: 0.5155, Accuracy: 99.81%

Test set: Average loss: 0.5242, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.518583
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.521431
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.512321
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.523672
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.517265
Train Epoch: 79 - Loss: 0.5151, Accuracy: 99.80%

Test set: Average loss: 0.5243, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.515646
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.545951
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.510431
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.511937
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.522028
Train Epoch: 80 - Loss: 0.5147, Accuracy: 99.79%

Test set: Average loss: 0.5232, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.508654
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.559922
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.520675
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.544132
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.516021
Train Epoch: 81 - Loss: 0.5143, Accuracy: 99.82%

Test set: Average loss: 0.5236, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.515302
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.510956
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.511802
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.516544
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.512707
Train Epoch: 82 - Loss: 0.5140, Accuracy: 99.82%

Test set: Average loss: 0.5231, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.516550
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.511266
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.511924
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.511359
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.509193
Train Epoch: 83 - Loss: 0.5135, Accuracy: 99.83%

Test set: Average loss: 0.5286, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.512580
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.518319
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.508720
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.509877
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.508799
Train Epoch: 84 - Loss: 0.5134, Accuracy: 99.82%

Test set: Average loss: 0.5224, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.507022
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.512782
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.519742
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.517084
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.508919
Train Epoch: 85 - Loss: 0.5129, Accuracy: 99.85%

Test set: Average loss: 0.5233, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.510973
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.510884
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.515255
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.515007
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.509751
Train Epoch: 86 - Loss: 0.5126, Accuracy: 99.85%

Test set: Average loss: 0.5227, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.514029
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.512956
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.507707
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.509389
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.514200
Train Epoch: 87 - Loss: 0.5123, Accuracy: 99.87%

Test set: Average loss: 0.5224, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.512969
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.512685
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.519778
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.505556
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.514949
Train Epoch: 88 - Loss: 0.5120, Accuracy: 99.87%

Test set: Average loss: 0.5234, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.517349
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.509675
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.509789
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.512881
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.534380
Train Epoch: 89 - Loss: 0.5117, Accuracy: 99.89%

Test set: Average loss: 0.5218, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.515215
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.507163
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.510451
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.509040
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.510273
Train Epoch: 90 - Loss: 0.5115, Accuracy: 99.89%

Test set: Average loss: 0.5220, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.520555
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.509952
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.515133
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.508022
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.510213
Train Epoch: 91 - Loss: 0.5113, Accuracy: 99.89%

Test set: Average loss: 0.5218, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.510989
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510558
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.511269
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.506430
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.510744
Train Epoch: 92 - Loss: 0.5111, Accuracy: 99.90%

Test set: Average loss: 0.5215, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.510294
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.507618
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.511599
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.509222
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.508344
Train Epoch: 93 - Loss: 0.5110, Accuracy: 99.90%

Test set: Average loss: 0.5213, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.508237
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.508884
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.508861
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.507438
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.510639
Train Epoch: 94 - Loss: 0.5109, Accuracy: 99.91%

Test set: Average loss: 0.5216, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.515632
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.506224
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.507768
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.516209
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.507632
Train Epoch: 95 - Loss: 0.5108, Accuracy: 99.90%

Test set: Average loss: 0.5215, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.509532
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.512036
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.512874
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.517220
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.506894
Train Epoch: 96 - Loss: 0.5106, Accuracy: 99.91%

Test set: Average loss: 0.5212, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.510257
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508725
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.511764
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.512632
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.508917
Train Epoch: 97 - Loss: 0.5106, Accuracy: 99.91%

Test set: Average loss: 0.5213, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506339
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.509099
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.505085
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.512460
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.508934
Train Epoch: 98 - Loss: 0.5105, Accuracy: 99.91%

Test set: Average loss: 0.5213, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.507941
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.508617
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.510032
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.507854
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.512347
Train Epoch: 99 - Loss: 0.5105, Accuracy: 99.91%

Test set: Average loss: 0.5213, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.516304
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.512336
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.509348
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.505870
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.508995
Train Epoch: 100 - Loss: 0.5105, Accuracy: 99.91%

Test set: Average loss: 0.5213, Accuracy: 9937/10000 (99.37%)


Training completed. Best Accuracy: 99.42%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.812809
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.830879
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.850172
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.655227
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.676982
Train Epoch: 1 - Loss: 0.8840, Accuracy: 85.58%

Test set: Average loss: 0.6385, Accuracy: 9642/10000 (96.42%)

New best model saved with accuracy: 96.42%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.657223
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.627871
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.615615
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.619427
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.624356
Train Epoch: 2 - Loss: 0.6175, Accuracy: 96.91%

Test set: Average loss: 0.5945, Accuracy: 9747/10000 (97.47%)

New best model saved with accuracy: 97.47%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.574946
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.563018
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.620573
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.625969
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.571703
Train Epoch: 3 - Loss: 0.5881, Accuracy: 97.87%

Test set: Average loss: 0.6069, Accuracy: 9731/10000 (97.31%)

Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.581786
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.571604
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.577660
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.602092
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.552503
Train Epoch: 4 - Loss: 0.5730, Accuracy: 98.37%

Test set: Average loss: 0.5699, Accuracy: 9830/10000 (98.30%)

New best model saved with accuracy: 98.30%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.565283
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.581863
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.567974
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.565835
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.565389
Train Epoch: 5 - Loss: 0.5641, Accuracy: 98.50%

Test set: Average loss: 0.5657, Accuracy: 9830/10000 (98.30%)

Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.561285
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.536735
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.603057
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.578451
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.567952
Train Epoch: 6 - Loss: 0.5573, Accuracy: 98.72%

Test set: Average loss: 0.5568, Accuracy: 9867/10000 (98.67%)

New best model saved with accuracy: 98.67%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.567775
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.544292
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.543543
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.532580
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.575687
Train Epoch: 7 - Loss: 0.5527, Accuracy: 98.83%

Test set: Average loss: 0.5524, Accuracy: 9870/10000 (98.70%)

New best model saved with accuracy: 98.70%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.532430
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.561213
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.579642
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.572102
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.581833
Train Epoch: 8 - Loss: 0.5488, Accuracy: 98.96%

Test set: Average loss: 0.5529, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.559562
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.568816
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.558032
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.535537
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.556016
Train Epoch: 9 - Loss: 0.5459, Accuracy: 99.01%

Test set: Average loss: 0.5479, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.538513
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.543942
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.526877
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.528323
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.539620
Train Epoch: 10 - Loss: 0.5429, Accuracy: 99.11%

Test set: Average loss: 0.5487, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.554407
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.531165
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.536961
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.543529
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.539834
Train Epoch: 11 - Loss: 0.5409, Accuracy: 99.16%

Test set: Average loss: 0.5439, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.554471
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.539201
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.582718
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.531371
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.517900
Train Epoch: 12 - Loss: 0.5389, Accuracy: 99.22%

Test set: Average loss: 0.5436, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.519825
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.532688
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.550170
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.525697
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.537504
Train Epoch: 13 - Loss: 0.5373, Accuracy: 99.27%

Test set: Average loss: 0.5413, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.518606
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.543664
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.530394
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.528831
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.526508
Train Epoch: 14 - Loss: 0.5359, Accuracy: 99.29%

Test set: Average loss: 0.5413, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.545544
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.550724
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.526169
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.526322
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.527107
Train Epoch: 15 - Loss: 0.5348, Accuracy: 99.32%

Test set: Average loss: 0.5406, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.531865
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.517702
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.527473
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.524969
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.523263
Train Epoch: 16 - Loss: 0.5337, Accuracy: 99.37%

Test set: Average loss: 0.5392, Accuracy: 9893/10000 (98.93%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.530747
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.524882
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.549996
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.538462
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.550356
Train Epoch: 17 - Loss: 0.5329, Accuracy: 99.40%

Test set: Average loss: 0.5388, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.555438
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.518077
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.532547
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.524853
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.542820
Train Epoch: 18 - Loss: 0.5322, Accuracy: 99.43%

Test set: Average loss: 0.5388, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.520771
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.540666
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.541740
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.517526
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.526556
Train Epoch: 19 - Loss: 0.5316, Accuracy: 99.46%

Test set: Average loss: 0.5388, Accuracy: 9892/10000 (98.92%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.540224
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.517872
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.547813
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.534202
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.542871
Train Epoch: 20 - Loss: 0.5311, Accuracy: 99.44%

Test set: Average loss: 0.5380, Accuracy: 9896/10000 (98.96%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.537519
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.520364
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.522424
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.531337
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.538789
Train Epoch: 21 - Loss: 0.5308, Accuracy: 99.46%

Test set: Average loss: 0.5380, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.518591
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.521229
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.537623
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.542091
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.532686
Train Epoch: 22 - Loss: 0.5305, Accuracy: 99.47%

Test set: Average loss: 0.5376, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.540307
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.519116
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.516380
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.524791
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.541454
Train Epoch: 23 - Loss: 0.5302, Accuracy: 99.47%

Test set: Average loss: 0.5377, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.528571
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.545795
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.532489
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.520332
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.523313
Train Epoch: 24 - Loss: 0.5301, Accuracy: 99.47%

Test set: Average loss: 0.5376, Accuracy: 9900/10000 (99.00%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.531238
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.526826
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.528760
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.526334
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.537081
Train Epoch: 25 - Loss: 0.5300, Accuracy: 99.47%

Test set: Average loss: 0.5375, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.525596
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.548172
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.546425
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.541172
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.529467
Train Epoch: 26 - Loss: 0.5357, Accuracy: 99.32%

Test set: Average loss: 0.5405, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.538333
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.528497
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.519632
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.520055
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.531810
Train Epoch: 27 - Loss: 0.5340, Accuracy: 99.32%

Test set: Average loss: 0.5386, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.531759
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.522875
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.534386
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.522808
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.537024
Train Epoch: 28 - Loss: 0.5326, Accuracy: 99.38%

Test set: Average loss: 0.5394, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.529691
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.518237
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.535878
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.542559
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.552326
Train Epoch: 29 - Loss: 0.5310, Accuracy: 99.41%

Test set: Average loss: 0.5390, Accuracy: 9894/10000 (98.94%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.516751
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.517626
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.521756
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.536033
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.518313
Train Epoch: 30 - Loss: 0.5298, Accuracy: 99.43%

Test set: Average loss: 0.5428, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.530688
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.542526
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.518394
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.537401
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.538046
Train Epoch: 31 - Loss: 0.5286, Accuracy: 99.49%

Test set: Average loss: 0.5350, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.549748
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.537458
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.521779
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.532641
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.525918
Train Epoch: 32 - Loss: 0.5276, Accuracy: 99.51%

Test set: Average loss: 0.5347, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.524477
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.565778
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.526663
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.527487
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.523917
Train Epoch: 33 - Loss: 0.5264, Accuracy: 99.55%

Test set: Average loss: 0.5329, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.522333
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.548559
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.518668
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.519658
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.516915
Train Epoch: 34 - Loss: 0.5254, Accuracy: 99.57%

Test set: Average loss: 0.5332, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.518202
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.518719
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.524956
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.517269
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.526429
Train Epoch: 35 - Loss: 0.5247, Accuracy: 99.60%

Test set: Average loss: 0.5341, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.520986
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.551245
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.520214
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.532852
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.537279
Train Epoch: 36 - Loss: 0.5240, Accuracy: 99.61%

Test set: Average loss: 0.5319, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.531706
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.521189
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.534599
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.540379
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.527489
Train Epoch: 37 - Loss: 0.5234, Accuracy: 99.64%

Test set: Average loss: 0.5318, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.516094
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.543050
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.517923
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.523781
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.518371
Train Epoch: 38 - Loss: 0.5227, Accuracy: 99.64%

Test set: Average loss: 0.5312, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.526489
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.514217
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.529229
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.523230
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.512186
Train Epoch: 39 - Loss: 0.5221, Accuracy: 99.67%

Test set: Average loss: 0.5305, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.511310
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.521930
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.514478
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.519330
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.520784
Train Epoch: 40 - Loss: 0.5216, Accuracy: 99.67%

Test set: Average loss: 0.5308, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.514365
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.515149
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.521007
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.524791
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.516209
Train Epoch: 41 - Loss: 0.5211, Accuracy: 99.69%

Test set: Average loss: 0.5306, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.519596
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.536547
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.522837
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.528026
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.525866
Train Epoch: 42 - Loss: 0.5208, Accuracy: 99.70%

Test set: Average loss: 0.5301, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.513899
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.524909
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.542735
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.522375
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.514414
Train Epoch: 43 - Loss: 0.5204, Accuracy: 99.70%

Test set: Average loss: 0.5305, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.516884
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.524415
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.516033
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.517346
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.513209
Train Epoch: 44 - Loss: 0.5201, Accuracy: 99.72%

Test set: Average loss: 0.5303, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.524943
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.532523
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.516345
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.514079
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.518871
Train Epoch: 45 - Loss: 0.5199, Accuracy: 99.72%

Test set: Average loss: 0.5300, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.519732
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.520420
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.514778
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.520797
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.525580
Train Epoch: 46 - Loss: 0.5197, Accuracy: 99.73%

Test set: Average loss: 0.5299, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.519477
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.517769
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.511970
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.517145
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.510448
Train Epoch: 47 - Loss: 0.5196, Accuracy: 99.73%

Test set: Average loss: 0.5297, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.516396
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.519349
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.516529
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.521865
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.524834
Train Epoch: 48 - Loss: 0.5194, Accuracy: 99.73%

Test set: Average loss: 0.5296, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.520959
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.522434
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.515454
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.516468
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.522000
Train Epoch: 49 - Loss: 0.5194, Accuracy: 99.73%

Test set: Average loss: 0.5296, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.518003
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.525252
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.519432
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.513132
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.508204
Train Epoch: 50 - Loss: 0.5193, Accuracy: 99.73%

Test set: Average loss: 0.5296, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.512777
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.534876
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.516074
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.517472
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.525790
Train Epoch: 51 - Loss: 0.5227, Accuracy: 99.65%

Test set: Average loss: 0.5314, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.523474
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.516705
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.523296
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.514966
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.533791
Train Epoch: 52 - Loss: 0.5225, Accuracy: 99.64%

Test set: Average loss: 0.5299, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.509496
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.514303
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.518241
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.517396
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.517414
Train Epoch: 53 - Loss: 0.5217, Accuracy: 99.66%

Test set: Average loss: 0.5312, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.520080
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.535486
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.513853
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.513650
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.513258
Train Epoch: 54 - Loss: 0.5212, Accuracy: 99.67%

Test set: Average loss: 0.5292, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.518390
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.534543
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.535814
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.535211
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.514835
Train Epoch: 55 - Loss: 0.5207, Accuracy: 99.68%

Test set: Average loss: 0.5297, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.536146
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.517498
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.528480
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.529887
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.514280
Train Epoch: 56 - Loss: 0.5201, Accuracy: 99.68%

Test set: Average loss: 0.5290, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.512978
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.523886
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.517447
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.513810
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.547115
Train Epoch: 57 - Loss: 0.5194, Accuracy: 99.71%

Test set: Average loss: 0.5292, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.511050
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.519447
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.516413
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.532439
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.509864
Train Epoch: 58 - Loss: 0.5189, Accuracy: 99.74%

Test set: Average loss: 0.5288, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.513057
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.515031
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.516063
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.530625
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.515330
Train Epoch: 59 - Loss: 0.5185, Accuracy: 99.73%

Test set: Average loss: 0.5275, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.511736
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.523548
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.510348
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.517165
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.514761
Train Epoch: 60 - Loss: 0.5180, Accuracy: 99.75%

Test set: Average loss: 0.5272, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.510497
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.534498
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.515114
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.511972
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.516584
Train Epoch: 61 - Loss: 0.5175, Accuracy: 99.75%

Test set: Average loss: 0.5280, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.511762
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.515815
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.512777
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.510054
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.516774
Train Epoch: 62 - Loss: 0.5171, Accuracy: 99.78%

Test set: Average loss: 0.5274, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.540754
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.513895
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.525672
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.512019
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.512265
Train Epoch: 63 - Loss: 0.5167, Accuracy: 99.80%

Test set: Average loss: 0.5278, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.510821
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.529511
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.529042
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.519266
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.511381
Train Epoch: 64 - Loss: 0.5162, Accuracy: 99.79%

Test set: Average loss: 0.5269, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.527007
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.510374
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.510302
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.510981
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.514389
Train Epoch: 65 - Loss: 0.5160, Accuracy: 99.81%

Test set: Average loss: 0.5274, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.511798
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.515981
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.515557
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.515395
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.520231
Train Epoch: 66 - Loss: 0.5157, Accuracy: 99.82%

Test set: Average loss: 0.5267, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.516334
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.514469
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.509976
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.509386
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.520745
Train Epoch: 67 - Loss: 0.5154, Accuracy: 99.83%

Test set: Average loss: 0.5267, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.511264
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.514558
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.511004
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.516892
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.528266
Train Epoch: 68 - Loss: 0.5152, Accuracy: 99.82%

Test set: Average loss: 0.5265, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.508320
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.510674
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.512779
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.509268
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.513919
Train Epoch: 69 - Loss: 0.5151, Accuracy: 99.83%

Test set: Average loss: 0.5262, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.512298
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.516913
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.513377
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.513587
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.526849
Train Epoch: 70 - Loss: 0.5149, Accuracy: 99.83%

Test set: Average loss: 0.5266, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.524284
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.515550
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.541277
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.508230
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.519690
Train Epoch: 71 - Loss: 0.5148, Accuracy: 99.83%

Test set: Average loss: 0.5262, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.511138
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.516918
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509794
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.509727
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.507483
Train Epoch: 72 - Loss: 0.5147, Accuracy: 99.84%

Test set: Average loss: 0.5263, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.529391
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.523102
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.514091
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.519653
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.513043
Train Epoch: 73 - Loss: 0.5146, Accuracy: 99.84%

Test set: Average loss: 0.5263, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.512588
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.507204
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.509037
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.544193
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.511562
Train Epoch: 74 - Loss: 0.5145, Accuracy: 99.84%

Test set: Average loss: 0.5262, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.510186
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.507464
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.507842
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.527071
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.509901
Train Epoch: 75 - Loss: 0.5145, Accuracy: 99.84%

Test set: Average loss: 0.5262, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.511620
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.525300
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.516868
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.515193
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.522553
Train Epoch: 76 - Loss: 0.5171, Accuracy: 99.79%

Test set: Average loss: 0.5316, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.517261
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.511273
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.532741
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.514218
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.519218
Train Epoch: 77 - Loss: 0.5169, Accuracy: 99.79%

Test set: Average loss: 0.5270, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.517434
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.516984
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.523316
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.509358
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.522828
Train Epoch: 78 - Loss: 0.5165, Accuracy: 99.79%

Test set: Average loss: 0.5271, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.517980
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.513577
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.524552
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.520935
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.520706
Train Epoch: 79 - Loss: 0.5163, Accuracy: 99.78%

Test set: Average loss: 0.5264, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.512194
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.520131
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.522096
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.510284
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.512965
Train Epoch: 80 - Loss: 0.5158, Accuracy: 99.80%

Test set: Average loss: 0.5268, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.525843
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.510808
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.516019
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.517460
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.514982
Train Epoch: 81 - Loss: 0.5155, Accuracy: 99.79%

Test set: Average loss: 0.5273, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.516678
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.510904
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.508785
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.511538
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.516798
Train Epoch: 82 - Loss: 0.5150, Accuracy: 99.82%

Test set: Average loss: 0.5269, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.511937
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.510640
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.508613
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.517699
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.508500
Train Epoch: 83 - Loss: 0.5147, Accuracy: 99.83%

Test set: Average loss: 0.5366, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.519388
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.512226
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.508973
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.519618
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.523928
Train Epoch: 84 - Loss: 0.5144, Accuracy: 99.83%

Test set: Average loss: 0.5297, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.517344
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.508317
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.512553
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.514585
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.514621
Train Epoch: 85 - Loss: 0.5139, Accuracy: 99.83%

Test set: Average loss: 0.5280, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.513056
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.507686
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.531321
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.506799
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.510116
Train Epoch: 86 - Loss: 0.5137, Accuracy: 99.85%

Test set: Average loss: 0.5261, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.515387
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.507926
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.519927
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.508164
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.512270
Train Epoch: 87 - Loss: 0.5133, Accuracy: 99.87%

Test set: Average loss: 0.5266, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.510148
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.517420
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.507602
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.515324
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.513052
Train Epoch: 88 - Loss: 0.5131, Accuracy: 99.86%

Test set: Average loss: 0.5253, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.510565
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.518063
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.507591
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.513289
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.510541
Train Epoch: 89 - Loss: 0.5128, Accuracy: 99.87%

Test set: Average loss: 0.5244, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.516157
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513700
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.513970
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.510464
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.509512
Train Epoch: 90 - Loss: 0.5126, Accuracy: 99.87%

Test set: Average loss: 0.5250, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.513476
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.514975
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.512591
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.511301
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.507357
Train Epoch: 91 - Loss: 0.5124, Accuracy: 99.89%

Test set: Average loss: 0.5246, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.516609
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510719
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510635
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.508659
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.512783
Train Epoch: 92 - Loss: 0.5122, Accuracy: 99.89%

Test set: Average loss: 0.5250, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.512242
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.508654
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.508020
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.515822
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.512148
Train Epoch: 93 - Loss: 0.5120, Accuracy: 99.90%

Test set: Average loss: 0.5242, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.507195
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.519315
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.512655
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.514754
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.511275
Train Epoch: 94 - Loss: 0.5119, Accuracy: 99.89%

Test set: Average loss: 0.5242, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.508524
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.509852
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.511348
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.508476
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.512043
Train Epoch: 95 - Loss: 0.5118, Accuracy: 99.90%

Test set: Average loss: 0.5247, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.508155
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.507837
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507582
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508434
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.506948
Train Epoch: 96 - Loss: 0.5117, Accuracy: 99.91%

Test set: Average loss: 0.5243, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.515321
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.507769
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.532279
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.510335
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.508113
Train Epoch: 97 - Loss: 0.5116, Accuracy: 99.91%

Test set: Average loss: 0.5244, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.520860
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.508559
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.507006
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.516041
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.512230
Train Epoch: 98 - Loss: 0.5116, Accuracy: 99.91%

Test set: Average loss: 0.5243, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.531977
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.519883
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.509576
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.509808
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.516279
Train Epoch: 99 - Loss: 0.5115, Accuracy: 99.91%

Test set: Average loss: 0.5243, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.517485
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.518267
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.506823
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.518082
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.514625
Train Epoch: 100 - Loss: 0.5115, Accuracy: 99.92%

Test set: Average loss: 0.5243, Accuracy: 9926/10000 (99.26%)


Training completed. Best Accuracy: 99.30%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.522477
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.799035
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.722528
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.667661
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.629691
Train Epoch: 1 - Loss: 0.8633, Accuracy: 86.50%

Test set: Average loss: 0.6243, Accuracy: 9676/10000 (96.76%)

New best model saved with accuracy: 96.76%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.584987
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.620481
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.635607
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.586884
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.588494
Train Epoch: 2 - Loss: 0.6089, Accuracy: 97.20%

Test set: Average loss: 0.5878, Accuracy: 9781/10000 (97.81%)

New best model saved with accuracy: 97.81%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.595853
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.561046
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.580934
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.592157
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.582814
Train Epoch: 3 - Loss: 0.5817, Accuracy: 98.04%

Test set: Average loss: 0.5743, Accuracy: 9824/10000 (98.24%)

New best model saved with accuracy: 98.24%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.561304
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.608576
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.570144
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.570493
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.550315
Train Epoch: 4 - Loss: 0.5684, Accuracy: 98.41%

Test set: Average loss: 0.5602, Accuracy: 9846/10000 (98.46%)

New best model saved with accuracy: 98.46%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.548792
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.544936
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.568986
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.565144
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.567309
Train Epoch: 5 - Loss: 0.5598, Accuracy: 98.60%

Test set: Average loss: 0.5573, Accuracy: 9859/10000 (98.59%)

New best model saved with accuracy: 98.59%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.554441
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.534556
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.569854
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.574715
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.548418
Train Epoch: 6 - Loss: 0.5539, Accuracy: 98.81%

Test set: Average loss: 0.5513, Accuracy: 9872/10000 (98.72%)

New best model saved with accuracy: 98.72%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.568983
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.533270
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.555050
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.573324
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.555348
Train Epoch: 7 - Loss: 0.5493, Accuracy: 98.94%

Test set: Average loss: 0.5484, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.549550
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.531901
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.546368
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.541277
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.552901
Train Epoch: 8 - Loss: 0.5459, Accuracy: 99.02%

Test set: Average loss: 0.5468, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.534981
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.568085
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.537885
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.528887
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.524255
Train Epoch: 9 - Loss: 0.5431, Accuracy: 99.08%

Test set: Average loss: 0.5466, Accuracy: 9887/10000 (98.87%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.532601
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.563626
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.556025
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.548752
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.537467
Train Epoch: 10 - Loss: 0.5409, Accuracy: 99.17%

Test set: Average loss: 0.5425, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.542828
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.564235
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.558694
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.542336
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.522513
Train Epoch: 11 - Loss: 0.5387, Accuracy: 99.21%

Test set: Average loss: 0.5418, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.525648
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.532412
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.568865
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.535889
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.533715
Train Epoch: 12 - Loss: 0.5371, Accuracy: 99.25%

Test set: Average loss: 0.5402, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.526324
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.566511
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.533754
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.517692
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.518638
Train Epoch: 13 - Loss: 0.5355, Accuracy: 99.28%

Test set: Average loss: 0.5392, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.568074
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.532961
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.535601
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.529661
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.540621
Train Epoch: 14 - Loss: 0.5342, Accuracy: 99.33%

Test set: Average loss: 0.5385, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.535102
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.527780
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.526621
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.533923
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.534868
Train Epoch: 15 - Loss: 0.5332, Accuracy: 99.35%

Test set: Average loss: 0.5374, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.517598
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.536396
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.544988
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.539240
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.554549
Train Epoch: 16 - Loss: 0.5324, Accuracy: 99.35%

Test set: Average loss: 0.5364, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.515777
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.514389
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.545009
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.521627
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.525192
Train Epoch: 17 - Loss: 0.5315, Accuracy: 99.40%

Test set: Average loss: 0.5361, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.529663
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.524036
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.518293
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.526371
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.530486
Train Epoch: 18 - Loss: 0.5308, Accuracy: 99.42%

Test set: Average loss: 0.5364, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.530816
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.571873
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.518582
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.559841
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.512272
Train Epoch: 19 - Loss: 0.5303, Accuracy: 99.44%

Test set: Average loss: 0.5359, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.525615
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.538319
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.546259
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.531329
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.518305
Train Epoch: 20 - Loss: 0.5298, Accuracy: 99.42%

Test set: Average loss: 0.5357, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.514812
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.551001
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.530073
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.547646
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.529610
Train Epoch: 21 - Loss: 0.5294, Accuracy: 99.45%

Test set: Average loss: 0.5355, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.537488
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.531885
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.527266
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.552133
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.515215
Train Epoch: 22 - Loss: 0.5292, Accuracy: 99.46%

Test set: Average loss: 0.5351, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.537615
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.532065
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.520574
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.526996
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.519248
Train Epoch: 23 - Loss: 0.5289, Accuracy: 99.46%

Test set: Average loss: 0.5350, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.517740
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.553365
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.527886
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.528071
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.543100
Train Epoch: 24 - Loss: 0.5288, Accuracy: 99.47%

Test set: Average loss: 0.5350, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.523403
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.521154
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.513730
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.531547
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.520950
Train Epoch: 25 - Loss: 0.5287, Accuracy: 99.47%

Test set: Average loss: 0.5350, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.548646
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.533865
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.537961
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.531187
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.523113
Train Epoch: 26 - Loss: 0.5345, Accuracy: 99.31%

Test set: Average loss: 0.5468, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.555810
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.547042
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.523932
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.532307
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.551450
Train Epoch: 27 - Loss: 0.5325, Accuracy: 99.35%

Test set: Average loss: 0.5379, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.537186
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.519747
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.519334
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.512685
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.523001
Train Epoch: 28 - Loss: 0.5311, Accuracy: 99.38%

Test set: Average loss: 0.5352, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.522178
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.523531
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.522786
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.528056
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.546124
Train Epoch: 29 - Loss: 0.5301, Accuracy: 99.40%

Test set: Average loss: 0.5387, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.542940
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.537458
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.530861
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.537179
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.543395
Train Epoch: 30 - Loss: 0.5288, Accuracy: 99.47%

Test set: Average loss: 0.5347, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.516769
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.517547
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.517709
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.524596
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.514885
Train Epoch: 31 - Loss: 0.5275, Accuracy: 99.48%

Test set: Average loss: 0.5362, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.533620
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.528044
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.530359
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.520485
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.526514
Train Epoch: 32 - Loss: 0.5267, Accuracy: 99.50%

Test set: Average loss: 0.5308, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.520763
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.524884
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.524442
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.525851
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.521833
Train Epoch: 33 - Loss: 0.5256, Accuracy: 99.53%

Test set: Average loss: 0.5335, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.537566
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.529264
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.520426
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.512972
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.524511
Train Epoch: 34 - Loss: 0.5249, Accuracy: 99.54%

Test set: Average loss: 0.5339, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.549277
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.528143
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.526170
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.518538
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.520171
Train Epoch: 35 - Loss: 0.5239, Accuracy: 99.58%

Test set: Average loss: 0.5306, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.526024
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.515125
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.512491
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.517749
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.529005
Train Epoch: 36 - Loss: 0.5231, Accuracy: 99.60%

Test set: Average loss: 0.5311, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.523395
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.513827
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.519945
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.532642
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.541221
Train Epoch: 37 - Loss: 0.5225, Accuracy: 99.60%

Test set: Average loss: 0.5297, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.524090
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.511884
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.513838
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.514750
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.511687
Train Epoch: 38 - Loss: 0.5219, Accuracy: 99.65%

Test set: Average loss: 0.5304, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.517431
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.513809
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.526712
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.513899
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.530183
Train Epoch: 39 - Loss: 0.5213, Accuracy: 99.67%

Test set: Average loss: 0.5285, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.522575
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.526208
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.540381
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.542567
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.523907
Train Epoch: 40 - Loss: 0.5208, Accuracy: 99.67%

Test set: Average loss: 0.5290, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.515785
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.520416
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.516533
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.512797
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.524227
Train Epoch: 41 - Loss: 0.5203, Accuracy: 99.69%

Test set: Average loss: 0.5289, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.520242
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.510146
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.516824
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.514208
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.538308
Train Epoch: 42 - Loss: 0.5200, Accuracy: 99.70%

Test set: Average loss: 0.5277, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.513126
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.511569
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.514223
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.510328
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.516043
Train Epoch: 43 - Loss: 0.5196, Accuracy: 99.72%

Test set: Average loss: 0.5279, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.514090
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.508918
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.527697
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.517023
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.526646
Train Epoch: 44 - Loss: 0.5193, Accuracy: 99.72%

Test set: Average loss: 0.5280, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.528841
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.515624
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.525376
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.512765
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.532649
Train Epoch: 45 - Loss: 0.5190, Accuracy: 99.72%

Test set: Average loss: 0.5277, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.514268
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.521294
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.523892
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.516350
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.512713
Train Epoch: 46 - Loss: 0.5189, Accuracy: 99.73%

Test set: Average loss: 0.5276, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.515876
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.508928
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.513164
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.510607
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.515027
Train Epoch: 47 - Loss: 0.5187, Accuracy: 99.74%

Test set: Average loss: 0.5276, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.510531
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.511279
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.516620
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.523424
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.518904
Train Epoch: 48 - Loss: 0.5186, Accuracy: 99.75%

Test set: Average loss: 0.5276, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.517877
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.515336
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.515546
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.514421
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.525990
Train Epoch: 49 - Loss: 0.5185, Accuracy: 99.75%

Test set: Average loss: 0.5275, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.518162
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.515770
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.527366
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.511927
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.511731
Train Epoch: 50 - Loss: 0.5185, Accuracy: 99.75%

Test set: Average loss: 0.5275, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.516536
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.529101
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.529656
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.516292
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.514209
Train Epoch: 51 - Loss: 0.5223, Accuracy: 99.61%

Test set: Average loss: 0.5308, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.522784
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.514898
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.515276
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.525834
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.513527
Train Epoch: 52 - Loss: 0.5218, Accuracy: 99.63%

Test set: Average loss: 0.5289, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.519654
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.514800
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.526756
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.519697
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.528194
Train Epoch: 53 - Loss: 0.5210, Accuracy: 99.64%

Test set: Average loss: 0.5290, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.513505
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.531479
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.535756
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.524984
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.517502
Train Epoch: 54 - Loss: 0.5204, Accuracy: 99.66%

Test set: Average loss: 0.5283, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.511617
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.519666
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.513155
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.510784
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.521219
Train Epoch: 55 - Loss: 0.5200, Accuracy: 99.67%

Test set: Average loss: 0.5279, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.513429
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.518116
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.527166
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.521946
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.515943
Train Epoch: 56 - Loss: 0.5191, Accuracy: 99.72%

Test set: Average loss: 0.5293, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.512506
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.522691
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.517540
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.551188
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.510582
Train Epoch: 57 - Loss: 0.5186, Accuracy: 99.72%

Test set: Average loss: 0.5295, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.520386
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.525230
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.525860
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.513987
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.519169
Train Epoch: 58 - Loss: 0.5181, Accuracy: 99.74%

Test set: Average loss: 0.5292, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.522505
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.511257
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.519166
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.524779
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.514317
Train Epoch: 59 - Loss: 0.5177, Accuracy: 99.73%

Test set: Average loss: 0.5295, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.513462
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.518766
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.518693
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.514724
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.517236
Train Epoch: 60 - Loss: 0.5171, Accuracy: 99.75%

Test set: Average loss: 0.5287, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.524621
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.526101
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.518049
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.537202
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.531615
Train Epoch: 61 - Loss: 0.5167, Accuracy: 99.77%

Test set: Average loss: 0.5263, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.515935
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.510474
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.517141
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.508925
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.509621
Train Epoch: 62 - Loss: 0.5162, Accuracy: 99.80%

Test set: Average loss: 0.5292, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.531477
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.511183
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.539682
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.516844
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.513411
Train Epoch: 63 - Loss: 0.5159, Accuracy: 99.80%

Test set: Average loss: 0.5253, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.513818
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.512070
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.514625
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.509101
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.520217
Train Epoch: 64 - Loss: 0.5155, Accuracy: 99.81%

Test set: Average loss: 0.5254, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.511172
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.509191
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.511478
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.510458
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.514005
Train Epoch: 65 - Loss: 0.5151, Accuracy: 99.81%

Test set: Average loss: 0.5251, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.515863
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.514775
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.508666
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.511502
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.516433
Train Epoch: 66 - Loss: 0.5148, Accuracy: 99.83%

Test set: Average loss: 0.5255, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.515816
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.510229
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.518480
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.520400
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513863
Train Epoch: 67 - Loss: 0.5146, Accuracy: 99.83%

Test set: Average loss: 0.5252, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.510076
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.511907
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.514278
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.517624
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.511842
Train Epoch: 68 - Loss: 0.5144, Accuracy: 99.83%

Test set: Average loss: 0.5255, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.522240
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.510990
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.512486
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.509883
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.509391
Train Epoch: 69 - Loss: 0.5142, Accuracy: 99.85%

Test set: Average loss: 0.5251, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.512032
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.507720
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.521604
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.533935
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.513634
Train Epoch: 70 - Loss: 0.5140, Accuracy: 99.85%

Test set: Average loss: 0.5250, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.508875
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.521263
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.530430
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.516935
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.528154
Train Epoch: 71 - Loss: 0.5139, Accuracy: 99.85%

Test set: Average loss: 0.5251, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.510326
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.514199
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.510274
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.513185
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.510529
Train Epoch: 72 - Loss: 0.5138, Accuracy: 99.85%

Test set: Average loss: 0.5248, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.511415
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508326
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.509071
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.520683
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.507441
Train Epoch: 73 - Loss: 0.5137, Accuracy: 99.85%

Test set: Average loss: 0.5248, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.510124
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.513013
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.510654
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.525882
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.515573
Train Epoch: 74 - Loss: 0.5137, Accuracy: 99.86%

Test set: Average loss: 0.5248, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.511725
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.514071
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.508703
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.512135
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.510893
Train Epoch: 75 - Loss: 0.5136, Accuracy: 99.86%

Test set: Average loss: 0.5248, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.519657
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.512066
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.517518
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.517464
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.517772
Train Epoch: 76 - Loss: 0.5163, Accuracy: 99.79%

Test set: Average loss: 0.5268, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.510596
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.511928
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.517804
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.534920
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.507925
Train Epoch: 77 - Loss: 0.5161, Accuracy: 99.78%

Test set: Average loss: 0.5268, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.519154
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.525715
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.515684
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.515921
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.511255
Train Epoch: 78 - Loss: 0.5156, Accuracy: 99.81%

Test set: Average loss: 0.5270, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.518685
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.511611
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.512472
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.514302
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.515370
Train Epoch: 79 - Loss: 0.5154, Accuracy: 99.79%

Test set: Average loss: 0.5259, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.517665
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.509404
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.515347
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.521710
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.515751
Train Epoch: 80 - Loss: 0.5149, Accuracy: 99.83%

Test set: Average loss: 0.5250, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.518560
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.514904
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.510412
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.521024
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.512273
Train Epoch: 81 - Loss: 0.5146, Accuracy: 99.82%

Test set: Average loss: 0.5309, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.514230
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.516781
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.509815
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.518122
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.521461
Train Epoch: 82 - Loss: 0.5142, Accuracy: 99.83%

Test set: Average loss: 0.5252, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.522712
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.515953
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.510591
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.514914
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.521724
Train Epoch: 83 - Loss: 0.5138, Accuracy: 99.83%

Test set: Average loss: 0.5261, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.510249
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.520144
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.522664
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.511570
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.508662
Train Epoch: 84 - Loss: 0.5134, Accuracy: 99.85%

Test set: Average loss: 0.5244, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508030
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.513067
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.518854
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.506168
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.520844
Train Epoch: 85 - Loss: 0.5130, Accuracy: 99.88%

Test set: Average loss: 0.5260, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.515354
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.506898
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.511810
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.506921
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.508524
Train Epoch: 86 - Loss: 0.5128, Accuracy: 99.87%

Test set: Average loss: 0.5246, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.510586
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.507840
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.509757
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.517567
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.511753
Train Epoch: 87 - Loss: 0.5125, Accuracy: 99.86%

Test set: Average loss: 0.5250, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.515450
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.509409
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.518448
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.507164
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.509666
Train Epoch: 88 - Loss: 0.5122, Accuracy: 99.88%

Test set: Average loss: 0.5242, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.509385
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.509932
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.517799
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.511503
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.512067
Train Epoch: 89 - Loss: 0.5119, Accuracy: 99.88%

Test set: Average loss: 0.5237, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.510641
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.507393
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.509668
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.512980
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.505914
Train Epoch: 90 - Loss: 0.5117, Accuracy: 99.89%

Test set: Average loss: 0.5241, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.509039
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.512511
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.507163
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.506432
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.505954
Train Epoch: 91 - Loss: 0.5115, Accuracy: 99.89%

Test set: Average loss: 0.5237, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.511849
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.507130
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510731
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.513394
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.517467
Train Epoch: 92 - Loss: 0.5114, Accuracy: 99.89%

Test set: Average loss: 0.5242, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.510977
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.508507
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.508111
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.510019
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.515046
Train Epoch: 93 - Loss: 0.5112, Accuracy: 99.89%

Test set: Average loss: 0.5236, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.509246
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.507341
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.514030
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.515624
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.509946
Train Epoch: 94 - Loss: 0.5110, Accuracy: 99.91%

Test set: Average loss: 0.5234, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.509907
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.510497
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.507413
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.511264
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.508454
Train Epoch: 95 - Loss: 0.5109, Accuracy: 99.90%

Test set: Average loss: 0.5234, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.507938
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.507001
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.510796
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.509283
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.511328
Train Epoch: 96 - Loss: 0.5108, Accuracy: 99.91%

Test set: Average loss: 0.5234, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.509450
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509969
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.505934
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.508762
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507926
Train Epoch: 97 - Loss: 0.5108, Accuracy: 99.91%

Test set: Average loss: 0.5233, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.509000
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.517034
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.517705
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.506672
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.509968
Train Epoch: 98 - Loss: 0.5107, Accuracy: 99.90%

Test set: Average loss: 0.5233, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.505849
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.507573
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.509183
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.513751
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.508212
Train Epoch: 99 - Loss: 0.5107, Accuracy: 99.91%

Test set: Average loss: 0.5233, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.520981
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.508746
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508909
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.509635
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.517995
Train Epoch: 100 - Loss: 0.5106, Accuracy: 99.91%

Test set: Average loss: 0.5233, Accuracy: 9938/10000 (99.38%)


Training completed. Best Accuracy: 99.41%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 2.332285
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.829026
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.710047
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.685951
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.676428
Train Epoch: 1 - Loss: 0.8378, Accuracy: 88.02%

Test set: Average loss: 0.6165, Accuracy: 9720/10000 (97.20%)

New best model saved with accuracy: 97.20%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.594434
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.615075
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.592589
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.563556
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.558248
Train Epoch: 2 - Loss: 0.6050, Accuracy: 97.44%

Test set: Average loss: 0.5879, Accuracy: 9808/10000 (98.08%)

New best model saved with accuracy: 98.08%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.580167
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.594853
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.564372
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.585912
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.591983
Train Epoch: 3 - Loss: 0.5782, Accuracy: 98.11%

Test set: Average loss: 0.5642, Accuracy: 9844/10000 (98.44%)

New best model saved with accuracy: 98.44%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.546295
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.618222
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.582002
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.576136
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.541815
Train Epoch: 4 - Loss: 0.5645, Accuracy: 98.52%

Test set: Average loss: 0.5572, Accuracy: 9871/10000 (98.71%)

New best model saved with accuracy: 98.71%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.537642
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.540025
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.543122
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.551938
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.562226
Train Epoch: 5 - Loss: 0.5557, Accuracy: 98.75%

Test set: Average loss: 0.5567, Accuracy: 9875/10000 (98.75%)

New best model saved with accuracy: 98.75%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.554518
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.549284
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.557117
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.542090
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.539471
Train Epoch: 6 - Loss: 0.5499, Accuracy: 98.87%

Test set: Average loss: 0.5468, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.534702
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.547844
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.530641
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.565693
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.561122
Train Epoch: 7 - Loss: 0.5452, Accuracy: 99.07%

Test set: Average loss: 0.5438, Accuracy: 9884/10000 (98.84%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.547885
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.550646
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.566279
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.538671
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.523535
Train Epoch: 8 - Loss: 0.5419, Accuracy: 99.12%

Test set: Average loss: 0.5466, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.541183
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.522767
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.518837
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.529372
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.524814
Train Epoch: 9 - Loss: 0.5392, Accuracy: 99.18%

Test set: Average loss: 0.5420, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.524440
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.539741
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.542109
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.547551
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.542201
Train Epoch: 10 - Loss: 0.5367, Accuracy: 99.24%

Test set: Average loss: 0.5393, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.544067
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.525948
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.541950
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.572920
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.523949
Train Epoch: 11 - Loss: 0.5350, Accuracy: 99.29%

Test set: Average loss: 0.5388, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.538165
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.526563
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.525830
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.531334
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.534681
Train Epoch: 12 - Loss: 0.5333, Accuracy: 99.36%

Test set: Average loss: 0.5363, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.531899
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.524701
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.534442
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.523237
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.540942
Train Epoch: 13 - Loss: 0.5319, Accuracy: 99.38%

Test set: Average loss: 0.5347, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.529437
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.530605
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.518636
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.517704
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.529482
Train Epoch: 14 - Loss: 0.5307, Accuracy: 99.38%

Test set: Average loss: 0.5336, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.541236
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.527391
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.517519
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.524439
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.534331
Train Epoch: 15 - Loss: 0.5297, Accuracy: 99.44%

Test set: Average loss: 0.5342, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.519353
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.526377
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.545300
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.534636
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.528770
Train Epoch: 16 - Loss: 0.5289, Accuracy: 99.47%

Test set: Average loss: 0.5332, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.532535
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.537779
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.519657
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.521295
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.545434
Train Epoch: 17 - Loss: 0.5282, Accuracy: 99.48%

Test set: Average loss: 0.5327, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.532502
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.519201
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.530797
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.522107
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.533714
Train Epoch: 18 - Loss: 0.5275, Accuracy: 99.49%

Test set: Average loss: 0.5327, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.519062
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.524413
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.534804
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.526531
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.515217
Train Epoch: 19 - Loss: 0.5270, Accuracy: 99.52%

Test set: Average loss: 0.5321, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.546493
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.536785
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.522087
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.533665
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.522004
Train Epoch: 20 - Loss: 0.5266, Accuracy: 99.52%

Test set: Average loss: 0.5319, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.521650
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.520193
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.516234
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.513295
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.545746
Train Epoch: 21 - Loss: 0.5263, Accuracy: 99.53%

Test set: Average loss: 0.5316, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.524008
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.537927
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.520426
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.527356
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.521263
Train Epoch: 22 - Loss: 0.5261, Accuracy: 99.55%

Test set: Average loss: 0.5315, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.519736
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.523050
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.543570
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.522821
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.529312
Train Epoch: 23 - Loss: 0.5259, Accuracy: 99.56%

Test set: Average loss: 0.5315, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.547166
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.527499
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.537249
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.517276
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.524758
Train Epoch: 24 - Loss: 0.5258, Accuracy: 99.56%

Test set: Average loss: 0.5314, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.515749
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.533374
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.528636
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.523914
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.526050
Train Epoch: 25 - Loss: 0.5257, Accuracy: 99.56%

Test set: Average loss: 0.5314, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.528231
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.532756
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.524887
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.515870
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.523309
Train Epoch: 26 - Loss: 0.5306, Accuracy: 99.42%

Test set: Average loss: 0.5329, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.529497
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.517555
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.528362
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.527187
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.536212
Train Epoch: 27 - Loss: 0.5293, Accuracy: 99.42%

Test set: Average loss: 0.5325, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.527909
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.528780
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.514319
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.537193
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.521209
Train Epoch: 28 - Loss: 0.5279, Accuracy: 99.49%

Test set: Average loss: 0.5336, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.515924
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.544075
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.530956
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.534907
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.527357
Train Epoch: 29 - Loss: 0.5267, Accuracy: 99.50%

Test set: Average loss: 0.5309, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.521257
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.520486
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.518180
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.520647
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.526009
Train Epoch: 30 - Loss: 0.5255, Accuracy: 99.55%

Test set: Average loss: 0.5303, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.514490
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.515042
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.519229
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.523945
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.527950
Train Epoch: 31 - Loss: 0.5247, Accuracy: 99.54%

Test set: Average loss: 0.5296, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.520325
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.515509
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.522873
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.514259
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.516910
Train Epoch: 32 - Loss: 0.5238, Accuracy: 99.58%

Test set: Average loss: 0.5308, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.526422
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.530002
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.526931
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.516380
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.534006
Train Epoch: 33 - Loss: 0.5229, Accuracy: 99.61%

Test set: Average loss: 0.5286, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.518383
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.517402
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.526669
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.531503
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.529578
Train Epoch: 34 - Loss: 0.5222, Accuracy: 99.62%

Test set: Average loss: 0.5302, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.517125
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.509704
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.515802
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.518923
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.540822
Train Epoch: 35 - Loss: 0.5213, Accuracy: 99.64%

Test set: Average loss: 0.5280, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.519402
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.521162
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.517851
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.514876
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.517820
Train Epoch: 36 - Loss: 0.5205, Accuracy: 99.69%

Test set: Average loss: 0.5287, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.523924
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.515233
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.513425
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.514317
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.512770
Train Epoch: 37 - Loss: 0.5202, Accuracy: 99.68%

Test set: Average loss: 0.5281, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.532891
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.518918
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.529801
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.515453
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.514524
Train Epoch: 38 - Loss: 0.5195, Accuracy: 99.67%

Test set: Average loss: 0.5271, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.513932
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.516053
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.517125
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.514632
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.518956
Train Epoch: 39 - Loss: 0.5190, Accuracy: 99.71%

Test set: Average loss: 0.5263, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.518463
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.525661
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.523472
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.516561
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.518935
Train Epoch: 40 - Loss: 0.5187, Accuracy: 99.72%

Test set: Average loss: 0.5260, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.508493
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.514345
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.520332
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.517956
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.509182
Train Epoch: 41 - Loss: 0.5182, Accuracy: 99.74%

Test set: Average loss: 0.5263, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.520777
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.516822
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.521032
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.513666
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.515168
Train Epoch: 42 - Loss: 0.5179, Accuracy: 99.75%

Test set: Average loss: 0.5263, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.516531
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.524565
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.513862
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.514863
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.517686
Train Epoch: 43 - Loss: 0.5176, Accuracy: 99.76%

Test set: Average loss: 0.5257, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.514437
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.512616
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.514520
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.521935
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.516137
Train Epoch: 44 - Loss: 0.5174, Accuracy: 99.76%

Test set: Average loss: 0.5255, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.516002
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.532402
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.522841
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.516796
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.510115
Train Epoch: 45 - Loss: 0.5172, Accuracy: 99.77%

Test set: Average loss: 0.5256, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.514166
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.522137
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.516296
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.516354
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.541692
Train Epoch: 46 - Loss: 0.5170, Accuracy: 99.78%

Test set: Average loss: 0.5256, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.511385
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.513339
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.522190
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.514259
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.526348
Train Epoch: 47 - Loss: 0.5168, Accuracy: 99.77%

Test set: Average loss: 0.5254, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.511621
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.545768
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.515650
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.510877
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.532063
Train Epoch: 48 - Loss: 0.5167, Accuracy: 99.78%

Test set: Average loss: 0.5254, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.524050
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.515167
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.514866
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.515929
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.529700
Train Epoch: 49 - Loss: 0.5167, Accuracy: 99.78%

Test set: Average loss: 0.5254, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.512128
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.511209
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.521572
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.511574
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.523530
Train Epoch: 50 - Loss: 0.5166, Accuracy: 99.79%

Test set: Average loss: 0.5254, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.513294
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.528418
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.525917
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.514327
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.514583
Train Epoch: 51 - Loss: 0.5200, Accuracy: 99.68%

Test set: Average loss: 0.5278, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.527052
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.513048
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.518525
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.508596
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.511602
Train Epoch: 52 - Loss: 0.5195, Accuracy: 99.70%

Test set: Average loss: 0.5263, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.513074
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.515833
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.521155
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.521245
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.516027
Train Epoch: 53 - Loss: 0.5188, Accuracy: 99.70%

Test set: Average loss: 0.5254, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.516171
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.517119
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.511957
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.519224
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.520854
Train Epoch: 54 - Loss: 0.5183, Accuracy: 99.74%

Test set: Average loss: 0.5257, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.513621
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.514174
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.525551
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.511517
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.516342
Train Epoch: 55 - Loss: 0.5177, Accuracy: 99.73%

Test set: Average loss: 0.5258, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.523623
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.512529
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.517991
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.537852
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.508498
Train Epoch: 56 - Loss: 0.5173, Accuracy: 99.76%

Test set: Average loss: 0.5250, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.521692
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.514960
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.512721
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.540680
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.520258
Train Epoch: 57 - Loss: 0.5169, Accuracy: 99.75%

Test set: Average loss: 0.5250, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.508770
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.509428
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.513155
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.506894
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.519529
Train Epoch: 58 - Loss: 0.5163, Accuracy: 99.78%

Test set: Average loss: 0.5249, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.512889
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.522567
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.514126
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.507313
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.514894
Train Epoch: 59 - Loss: 0.5158, Accuracy: 99.81%

Test set: Average loss: 0.5249, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.515197
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.509462
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.519766
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.509169
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.513649
Train Epoch: 60 - Loss: 0.5154, Accuracy: 99.80%

Test set: Average loss: 0.5242, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.510034
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.509385
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.531251
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.514268
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.514907
Train Epoch: 61 - Loss: 0.5150, Accuracy: 99.81%

Test set: Average loss: 0.5238, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.513884
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.510906
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.511951
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.513544
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.510155
Train Epoch: 62 - Loss: 0.5146, Accuracy: 99.82%

Test set: Average loss: 0.5240, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.509326
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.517681
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.507073
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.520890
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.507946
Train Epoch: 63 - Loss: 0.5143, Accuracy: 99.83%

Test set: Average loss: 0.5245, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.515919
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.511088
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.509409
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.529126
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.510741
Train Epoch: 64 - Loss: 0.5140, Accuracy: 99.83%

Test set: Average loss: 0.5244, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.519023
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.508052
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.516425
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.516203
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.511754
Train Epoch: 65 - Loss: 0.5136, Accuracy: 99.84%

Test set: Average loss: 0.5237, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.520058
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.511879
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.517293
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.520678
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.508931
Train Epoch: 66 - Loss: 0.5134, Accuracy: 99.86%

Test set: Average loss: 0.5233, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.509290
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.511507
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.511905
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.512629
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.515934
Train Epoch: 67 - Loss: 0.5132, Accuracy: 99.86%

Test set: Average loss: 0.5230, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.510050
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.511580
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.516158
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.508181
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.516984
Train Epoch: 68 - Loss: 0.5130, Accuracy: 99.86%

Test set: Average loss: 0.5229, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.507253
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.516341
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.507111
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.514459
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.506080
Train Epoch: 69 - Loss: 0.5129, Accuracy: 99.86%

Test set: Average loss: 0.5230, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.513904
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.516611
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.511530
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.509984
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.508089
Train Epoch: 70 - Loss: 0.5128, Accuracy: 99.87%

Test set: Average loss: 0.5228, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.515371
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.513315
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.511891
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.507509
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.511310
Train Epoch: 71 - Loss: 0.5126, Accuracy: 99.87%

Test set: Average loss: 0.5228, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.513139
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509226
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.519606
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.511636
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.509180
Train Epoch: 72 - Loss: 0.5125, Accuracy: 99.88%

Test set: Average loss: 0.5227, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.507504
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.509094
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.507727
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.514724
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.510133
Train Epoch: 73 - Loss: 0.5125, Accuracy: 99.87%

Test set: Average loss: 0.5228, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.521424
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.508642
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.516101
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.506778
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513013
Train Epoch: 74 - Loss: 0.5124, Accuracy: 99.88%

Test set: Average loss: 0.5228, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.510132
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.506567
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.512907
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.509051
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.513802
Train Epoch: 75 - Loss: 0.5124, Accuracy: 99.88%

Test set: Average loss: 0.5227, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.517822
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.512394
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.513178
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.518416
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.512472
Train Epoch: 76 - Loss: 0.5148, Accuracy: 99.81%

Test set: Average loss: 0.5238, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.511005
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.511099
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.510181
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.510298
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.521684
Train Epoch: 77 - Loss: 0.5145, Accuracy: 99.83%

Test set: Average loss: 0.5233, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.510478
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.514273
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.522896
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.514134
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.525918
Train Epoch: 78 - Loss: 0.5140, Accuracy: 99.84%

Test set: Average loss: 0.5232, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.509653
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.520091
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.510214
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.510910
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.508245
Train Epoch: 79 - Loss: 0.5139, Accuracy: 99.83%

Test set: Average loss: 0.5232, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.513404
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.511746
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.512895
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.520716
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.518230
Train Epoch: 80 - Loss: 0.5134, Accuracy: 99.85%

Test set: Average loss: 0.5231, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.509604
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.511701
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.512963
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.509771
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.518281
Train Epoch: 81 - Loss: 0.5132, Accuracy: 99.86%

Test set: Average loss: 0.5235, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.508668
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.508119
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.512746
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.507830
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.512824
Train Epoch: 82 - Loss: 0.5129, Accuracy: 99.86%

Test set: Average loss: 0.5231, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.507341
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.506210
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.511350
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.514882
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.507546
Train Epoch: 83 - Loss: 0.5125, Accuracy: 99.87%

Test set: Average loss: 0.5223, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.531191
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.508921
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.510379
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.507200
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.513983
Train Epoch: 84 - Loss: 0.5121, Accuracy: 99.88%

Test set: Average loss: 0.5222, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.513977
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.509808
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.534482
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.513201
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.522463
Train Epoch: 85 - Loss: 0.5121, Accuracy: 99.87%

Test set: Average loss: 0.5231, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.508550
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.511736
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.509572
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.508934
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.516100
Train Epoch: 86 - Loss: 0.5118, Accuracy: 99.89%

Test set: Average loss: 0.5219, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.508188
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.521696
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.511406
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.508070
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.510081
Train Epoch: 87 - Loss: 0.5114, Accuracy: 99.89%

Test set: Average loss: 0.5222, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.515526
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.513343
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.511924
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.507957
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.508704
Train Epoch: 88 - Loss: 0.5112, Accuracy: 99.90%

Test set: Average loss: 0.5219, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.510169
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.507558
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.507837
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.515357
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.509578
Train Epoch: 89 - Loss: 0.5110, Accuracy: 99.91%

Test set: Average loss: 0.5212, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.511621
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513204
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.510104
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.509886
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.517895
Train Epoch: 90 - Loss: 0.5107, Accuracy: 99.91%

Test set: Average loss: 0.5229, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.505797
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.517313
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.507604
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.512231
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.515197
Train Epoch: 91 - Loss: 0.5106, Accuracy: 99.91%

Test set: Average loss: 0.5212, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.508865
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510872
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.514820
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.512080
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.517843
Train Epoch: 92 - Loss: 0.5104, Accuracy: 99.92%

Test set: Average loss: 0.5214, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.511667
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.516650
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.515861
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.505273
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.505553
Train Epoch: 93 - Loss: 0.5103, Accuracy: 99.92%

Test set: Average loss: 0.5210, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.506409
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.507662
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.515133
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.510694
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.518812
Train Epoch: 94 - Loss: 0.5102, Accuracy: 99.92%

Test set: Average loss: 0.5210, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.506350
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.507717
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506399
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.510254
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.506358
Train Epoch: 95 - Loss: 0.5101, Accuracy: 99.93%

Test set: Average loss: 0.5212, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.513811
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.507824
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507513
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508962
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.514623
Train Epoch: 96 - Loss: 0.5100, Accuracy: 99.92%

Test set: Average loss: 0.5210, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.512205
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.516382
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.506786
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.510798
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.504575
Train Epoch: 97 - Loss: 0.5099, Accuracy: 99.92%

Test set: Average loss: 0.5209, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.506092
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.508401
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.507732
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508042
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.532389
Train Epoch: 98 - Loss: 0.5099, Accuracy: 99.93%

Test set: Average loss: 0.5210, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.514085
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.508344
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.513492
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.512689
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.512812
Train Epoch: 99 - Loss: 0.5098, Accuracy: 99.93%

Test set: Average loss: 0.5210, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.508736
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.507987
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.506145
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.506378
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.508937
Train Epoch: 100 - Loss: 0.5098, Accuracy: 99.93%

Test set: Average loss: 0.5209, Accuracy: 9934/10000 (99.34%)


Training completed. Best Accuracy: 99.37%
