Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.294340
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.733464
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.730547
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.597846
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.592558
Train Epoch: 1 - Loss: 0.7018, Accuracy: 94.76%

Test set: Average loss: 0.5966, Accuracy: 9789/10000 (97.89%)

New best model saved with accuracy: 97.89%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.566340
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.591383
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.574498
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.588553
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.588470
Train Epoch: 2 - Loss: 0.5838, Accuracy: 98.19%

Test set: Average loss: 0.5645, Accuracy: 9850/10000 (98.50%)

New best model saved with accuracy: 98.50%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.581244
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.543881
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.574637
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.570480
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.568417
Train Epoch: 3 - Loss: 0.5658, Accuracy: 98.60%

Test set: Average loss: 0.5594, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.566851
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.557338
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.540720
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.570593
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.537481
Train Epoch: 4 - Loss: 0.5562, Accuracy: 98.89%

Test set: Average loss: 0.5500, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.532797
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.549621
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.525926
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.552683
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.535392
Train Epoch: 5 - Loss: 0.5507, Accuracy: 98.97%

Test set: Average loss: 0.5490, Accuracy: 9889/10000 (98.89%)

Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.544299
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.544707
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.542601
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.542398
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.567156
Train Epoch: 6 - Loss: 0.5456, Accuracy: 99.11%

Test set: Average loss: 0.5438, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.563891
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.521085
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.525235
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.544947
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.525838
Train Epoch: 7 - Loss: 0.5424, Accuracy: 99.21%

Test set: Average loss: 0.5398, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.546307
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.543422
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.532840
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.538154
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.539380
Train Epoch: 8 - Loss: 0.5394, Accuracy: 99.29%

Test set: Average loss: 0.5401, Accuracy: 9897/10000 (98.97%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.527256
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.566407
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.541861
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.543193
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.537667
Train Epoch: 9 - Loss: 0.5376, Accuracy: 99.32%

Test set: Average loss: 0.5358, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.533621
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.528527
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.532269
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.538561
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.528103
Train Epoch: 10 - Loss: 0.5356, Accuracy: 99.35%

Test set: Average loss: 0.5363, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.534055
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.548403
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.520303
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.536218
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.535796
Train Epoch: 11 - Loss: 0.5342, Accuracy: 99.42%

Test set: Average loss: 0.5342, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.518937
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.528911
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.539710
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.530052
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.531297
Train Epoch: 12 - Loss: 0.5329, Accuracy: 99.44%

Test set: Average loss: 0.5337, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.533891
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.534208
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.525613
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.529183
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.528644
Train Epoch: 13 - Loss: 0.5322, Accuracy: 99.45%

Test set: Average loss: 0.5326, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.532778
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.522720
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.529563
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.526798
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.538825
Train Epoch: 14 - Loss: 0.5307, Accuracy: 99.48%

Test set: Average loss: 0.5327, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.529495
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.525485
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.517897
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.519507
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.528286
Train Epoch: 15 - Loss: 0.5304, Accuracy: 99.52%

Test set: Average loss: 0.5315, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.521828
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.525815
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.538257
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.547340
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.532064
Train Epoch: 16 - Loss: 0.5290, Accuracy: 99.57%

Test set: Average loss: 0.5316, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.528561
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.522631
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.525762
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.516483
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.518492
Train Epoch: 17 - Loss: 0.5288, Accuracy: 99.55%

Test set: Average loss: 0.5305, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.527038
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.529653
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.523305
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.528256
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.538604
Train Epoch: 18 - Loss: 0.5280, Accuracy: 99.56%

Test set: Average loss: 0.5303, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.533083
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.518762
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.521089
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.517394
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.525865
Train Epoch: 19 - Loss: 0.5277, Accuracy: 99.59%

Test set: Average loss: 0.5302, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.522531
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.522051
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.541371
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.529233
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.519111
Train Epoch: 20 - Loss: 0.5276, Accuracy: 99.59%

Test set: Average loss: 0.5299, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.520807
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.536168
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.531648
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.516747
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.539119
Train Epoch: 21 - Loss: 0.5271, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.530542
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.529880
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.524656
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.511805
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.521206
Train Epoch: 22 - Loss: 0.5268, Accuracy: 99.60%

Test set: Average loss: 0.5296, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.551184
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.529288
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.527164
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.520629
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.536985
Train Epoch: 23 - Loss: 0.5270, Accuracy: 99.62%

Test set: Average loss: 0.5299, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.514675
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.530041
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.516271
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.518867
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.517043
Train Epoch: 24 - Loss: 0.5268, Accuracy: 99.61%

Test set: Average loss: 0.5296, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.530663
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.528652
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.528454
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.527186
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.527057
Train Epoch: 25 - Loss: 0.5266, Accuracy: 99.63%

Test set: Average loss: 0.5295, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.531143
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.525430
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.516411
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.531602
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.527598
Train Epoch: 26 - Loss: 0.5291, Accuracy: 99.52%

Test set: Average loss: 0.5305, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.517931
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.527273
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.533730
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.522085
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.545939
Train Epoch: 27 - Loss: 0.5280, Accuracy: 99.55%

Test set: Average loss: 0.5298, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.554203
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.533852
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.532695
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.529642
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.518587
Train Epoch: 28 - Loss: 0.5269, Accuracy: 99.58%

Test set: Average loss: 0.5278, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.514485
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.519024
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.535715
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.543567
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.518636
Train Epoch: 29 - Loss: 0.5261, Accuracy: 99.59%

Test set: Average loss: 0.5285, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.524923
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.526245
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.520593
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.515961
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.565135
Train Epoch: 30 - Loss: 0.5256, Accuracy: 99.56%

Test set: Average loss: 0.5283, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.522873
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.525197
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.513140
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.524743
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.514664
Train Epoch: 31 - Loss: 0.5244, Accuracy: 99.62%

Test set: Average loss: 0.5303, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.515251
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.520889
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.524746
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.515168
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.524427
Train Epoch: 32 - Loss: 0.5236, Accuracy: 99.64%

Test set: Average loss: 0.5270, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.518485
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.515936
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.519182
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.523264
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.519532
Train Epoch: 33 - Loss: 0.5231, Accuracy: 99.64%

Test set: Average loss: 0.5275, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.519026
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.524208
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.520034
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.519597
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.519069
Train Epoch: 34 - Loss: 0.5227, Accuracy: 99.68%

Test set: Average loss: 0.5268, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.514497
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.512506
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.519902
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.518100
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.520272
Train Epoch: 35 - Loss: 0.5217, Accuracy: 99.71%

Test set: Average loss: 0.5263, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.532313
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.512312
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.540344
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.515689
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.522496
Train Epoch: 36 - Loss: 0.5212, Accuracy: 99.73%

Test set: Average loss: 0.5256, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.525983
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.513011
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.530220
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.517774
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.530265
Train Epoch: 37 - Loss: 0.5208, Accuracy: 99.70%

Test set: Average loss: 0.5248, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.523653
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.514711
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.513724
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.509524
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.517475
Train Epoch: 38 - Loss: 0.5201, Accuracy: 99.74%

Test set: Average loss: 0.5251, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.522480
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.516765
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.517249
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.511516
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.528843
Train Epoch: 39 - Loss: 0.5199, Accuracy: 99.75%

Test set: Average loss: 0.5246, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.514408
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.532538
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.523385
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.520030
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.517076
Train Epoch: 40 - Loss: 0.5194, Accuracy: 99.76%

Test set: Average loss: 0.5245, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.509840
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.525448
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.523063
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.514460
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.510237
Train Epoch: 41 - Loss: 0.5194, Accuracy: 99.73%

Test set: Average loss: 0.5244, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.512360
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.517756
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.512657
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.514961
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.521069
Train Epoch: 42 - Loss: 0.5190, Accuracy: 99.77%

Test set: Average loss: 0.5245, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.517269
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.516961
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.515654
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.509711
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.511845
Train Epoch: 43 - Loss: 0.5185, Accuracy: 99.81%

Test set: Average loss: 0.5240, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.516792
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.513037
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.518588
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.545855
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.516676
Train Epoch: 44 - Loss: 0.5184, Accuracy: 99.78%

Test set: Average loss: 0.5239, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.516667
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.518072
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.515320
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.530582
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.516553
Train Epoch: 45 - Loss: 0.5183, Accuracy: 99.79%

Test set: Average loss: 0.5240, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.512937
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.521144
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.519644
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.513855
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.519238
Train Epoch: 46 - Loss: 0.5182, Accuracy: 99.80%

Test set: Average loss: 0.5240, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.515705
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.519719
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.517199
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.513288
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.511711
Train Epoch: 47 - Loss: 0.5179, Accuracy: 99.81%

Test set: Average loss: 0.5239, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.518974
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.510511
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.517999
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.516112
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.520104
Train Epoch: 48 - Loss: 0.5180, Accuracy: 99.81%

Test set: Average loss: 0.5240, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.512848
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.525482
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.520940
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.520946
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.511888
Train Epoch: 49 - Loss: 0.5178, Accuracy: 99.81%

Test set: Average loss: 0.5238, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.515452
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.515641
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.527062
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.511956
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.511855
Train Epoch: 50 - Loss: 0.5178, Accuracy: 99.81%

Test set: Average loss: 0.5238, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.512644
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.521558
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.523393
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.511827
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.514566
Train Epoch: 51 - Loss: 0.5197, Accuracy: 99.75%

Test set: Average loss: 0.5253, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.522064
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.524905
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.529959
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.521997
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.516354
Train Epoch: 52 - Loss: 0.5195, Accuracy: 99.74%

Test set: Average loss: 0.5246, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.517951
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.519076
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.512370
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.540068
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.532651
Train Epoch: 53 - Loss: 0.5191, Accuracy: 99.72%

Test set: Average loss: 0.5248, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.517731
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.512047
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.526627
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.526681
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.512518
Train Epoch: 54 - Loss: 0.5186, Accuracy: 99.76%

Test set: Average loss: 0.5240, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.518138
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.512194
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.512620
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.519321
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.520478
Train Epoch: 55 - Loss: 0.5181, Accuracy: 99.79%

Test set: Average loss: 0.5239, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.511150
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.531245
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.526110
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.510933
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.514982
Train Epoch: 56 - Loss: 0.5175, Accuracy: 99.80%

Test set: Average loss: 0.5244, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.514383
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.513541
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.531102
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.529351
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.512113
Train Epoch: 57 - Loss: 0.5176, Accuracy: 99.80%

Test set: Average loss: 0.5245, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.511009
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.512280
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.513078
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.510961
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.516938
Train Epoch: 58 - Loss: 0.5171, Accuracy: 99.80%

Test set: Average loss: 0.5228, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.515413
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.517370
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.521817
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.515612
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.519350
Train Epoch: 59 - Loss: 0.5165, Accuracy: 99.83%

Test set: Average loss: 0.5240, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.510807
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.515900
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.513099
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.513068
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.515958
Train Epoch: 60 - Loss: 0.5160, Accuracy: 99.84%

Test set: Average loss: 0.5227, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.520020
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.512022
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.510958
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.527232
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.515480
Train Epoch: 61 - Loss: 0.5162, Accuracy: 99.81%

Test set: Average loss: 0.5233, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.512350
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.519239
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.520897
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.515023
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.513647
Train Epoch: 62 - Loss: 0.5156, Accuracy: 99.85%

Test set: Average loss: 0.5222, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.520971
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.508963
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.511772
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.519816
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.515883
Train Epoch: 63 - Loss: 0.5154, Accuracy: 99.84%

Test set: Average loss: 0.5223, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.519227
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.523061
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.509728
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.520905
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.510266
Train Epoch: 64 - Loss: 0.5151, Accuracy: 99.86%

Test set: Average loss: 0.5217, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.515480
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.514190
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.508804
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.515231
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.515494
Train Epoch: 65 - Loss: 0.5150, Accuracy: 99.85%

Test set: Average loss: 0.5220, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.509463
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.512110
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.513926
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.509382
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.512476
Train Epoch: 66 - Loss: 0.5146, Accuracy: 99.88%

Test set: Average loss: 0.5213, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.510445
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.511758
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.509633
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.520217
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.509140
Train Epoch: 67 - Loss: 0.5145, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.516260
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.511007
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.509854
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.509471
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.515587
Train Epoch: 68 - Loss: 0.5144, Accuracy: 99.87%

Test set: Average loss: 0.5215, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.524741
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.512483
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.507875
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.517610
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.511368
Train Epoch: 69 - Loss: 0.5141, Accuracy: 99.89%

Test set: Average loss: 0.5214, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.513876
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.510220
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.507489
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.530293
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.510563
Train Epoch: 70 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5213, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.519420
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.510592
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.510924
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.508914
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.511626
Train Epoch: 71 - Loss: 0.5139, Accuracy: 99.89%

Test set: Average loss: 0.5214, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.511002
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.521302
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.525327
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.517688
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.514819
Train Epoch: 72 - Loss: 0.5139, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.507825
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.510424
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.511725
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.510966
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.515183
Train Epoch: 73 - Loss: 0.5140, Accuracy: 99.89%

Test set: Average loss: 0.5214, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.525812
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.518149
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.511271
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.520411
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513609
Train Epoch: 74 - Loss: 0.5139, Accuracy: 99.90%

Test set: Average loss: 0.5214, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.516338
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.510664
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.510201
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.511300
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.517605
Train Epoch: 75 - Loss: 0.5137, Accuracy: 99.91%

Test set: Average loss: 0.5216, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.508365
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.516512
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.514507
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.522087
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.512923
Train Epoch: 76 - Loss: 0.5154, Accuracy: 99.84%

Test set: Average loss: 0.5217, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.510937
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.513472
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.515976
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.513519
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.514418
Train Epoch: 77 - Loss: 0.5149, Accuracy: 99.87%

Test set: Average loss: 0.5216, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.521147
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.509805
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.509866
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.513528
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.532626
Train Epoch: 78 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5225, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.512236
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.514222
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.513064
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.507967
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.511094
Train Epoch: 79 - Loss: 0.5146, Accuracy: 99.84%

Test set: Average loss: 0.5225, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.517693
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.514359
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.512076
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.508929
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.515610
Train Epoch: 80 - Loss: 0.5142, Accuracy: 99.86%

Test set: Average loss: 0.5218, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.519783
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.511758
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.511202
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.510500
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.507736
Train Epoch: 81 - Loss: 0.5141, Accuracy: 99.89%

Test set: Average loss: 0.5207, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.513540
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.514599
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.510972
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.520261
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.520074
Train Epoch: 82 - Loss: 0.5140, Accuracy: 99.87%

Test set: Average loss: 0.5212, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.513087
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.509008
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.514616
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.516014
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.518373
Train Epoch: 83 - Loss: 0.5135, Accuracy: 99.89%

Test set: Average loss: 0.5206, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.519409
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.509308
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.513095
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.513560
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.510830
Train Epoch: 84 - Loss: 0.5132, Accuracy: 99.90%

Test set: Average loss: 0.5216, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508565
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.522040
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.508158
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.510418
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.506791
Train Epoch: 85 - Loss: 0.5128, Accuracy: 99.91%

Test set: Average loss: 0.5201, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.512429
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.512866
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.512760
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.518621
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.515417
Train Epoch: 86 - Loss: 0.5126, Accuracy: 99.91%

Test set: Average loss: 0.5216, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.538068
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.510346
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.512666
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.511906
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.507881
Train Epoch: 87 - Loss: 0.5125, Accuracy: 99.93%

Test set: Average loss: 0.5206, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.513043
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507994
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.521123
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.515049
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.509496
Train Epoch: 88 - Loss: 0.5124, Accuracy: 99.92%

Test set: Average loss: 0.5208, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.512216
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.516921
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.510943
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.511453
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.511471
Train Epoch: 89 - Loss: 0.5125, Accuracy: 99.91%

Test set: Average loss: 0.5198, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.508796
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.509554
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.510710
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.518601
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.519114
Train Epoch: 90 - Loss: 0.5121, Accuracy: 99.93%

Test set: Average loss: 0.5200, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.507263
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.508290
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.511891
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.508271
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.509602
Train Epoch: 91 - Loss: 0.5119, Accuracy: 99.93%

Test set: Average loss: 0.5200, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.506586
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510110
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.514589
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.511144
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.513377
Train Epoch: 92 - Loss: 0.5119, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.511248
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.511483
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.511482
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.507548
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.508442
Train Epoch: 93 - Loss: 0.5116, Accuracy: 99.93%

Test set: Average loss: 0.5197, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.511733
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.509654
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.509829
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.512717
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.511744
Train Epoch: 94 - Loss: 0.5116, Accuracy: 99.94%

Test set: Average loss: 0.5200, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.512537
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.510739
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.508252
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.513045
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.508539
Train Epoch: 95 - Loss: 0.5114, Accuracy: 99.92%

Test set: Average loss: 0.5196, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.509935
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.512835
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.514073
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508603
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.509407
Train Epoch: 96 - Loss: 0.5114, Accuracy: 99.94%

Test set: Average loss: 0.5197, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.512143
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508378
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.510173
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.512002
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507680
Train Epoch: 97 - Loss: 0.5113, Accuracy: 99.93%

Test set: Average loss: 0.5196, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.519420
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.508761
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.510687
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508226
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.512055
Train Epoch: 98 - Loss: 0.5114, Accuracy: 99.94%

Test set: Average loss: 0.5197, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.509412
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.517397
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.508675
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.511187
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.514576
Train Epoch: 99 - Loss: 0.5112, Accuracy: 99.94%

Test set: Average loss: 0.5196, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.509017
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.509141
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508869
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.514010
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.512288
Train Epoch: 100 - Loss: 0.5114, Accuracy: 99.94%

Test set: Average loss: 0.5198, Accuracy: 9947/10000 (99.47%)


Training completed. Best Accuracy: 99.52%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.431913
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.754217
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.662573
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.611979
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.577636
Train Epoch: 1 - Loss: 0.7262, Accuracy: 94.03%

Test set: Average loss: 0.5926, Accuracy: 9788/10000 (97.88%)

New best model saved with accuracy: 97.88%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.588735
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.569724
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.584531
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.562098
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.581315
Train Epoch: 2 - Loss: 0.5883, Accuracy: 98.15%

Test set: Average loss: 0.5760, Accuracy: 9831/10000 (98.31%)

New best model saved with accuracy: 98.31%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.585558
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.586090
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.554741
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.559002
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.543406
Train Epoch: 3 - Loss: 0.5683, Accuracy: 98.61%

Test set: Average loss: 0.5599, Accuracy: 9861/10000 (98.61%)

New best model saved with accuracy: 98.61%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.543658
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.550823
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.546285
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.579046
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.541566
Train Epoch: 4 - Loss: 0.5584, Accuracy: 98.88%

Test set: Average loss: 0.5610, Accuracy: 9865/10000 (98.65%)

New best model saved with accuracy: 98.65%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.569829
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.589042
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.574314
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.560924
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.543243
Train Epoch: 5 - Loss: 0.5518, Accuracy: 99.01%

Test set: Average loss: 0.5491, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.532483
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.535313
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.532510
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.542399
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.534464
Train Epoch: 6 - Loss: 0.5465, Accuracy: 99.13%

Test set: Average loss: 0.5465, Accuracy: 9891/10000 (98.91%)

New best model saved with accuracy: 98.91%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.547847
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.541133
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.526974
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.538617
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.527377
Train Epoch: 7 - Loss: 0.5432, Accuracy: 99.21%

Test set: Average loss: 0.5416, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.539088
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.551504
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.540840
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.551853
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.540860
Train Epoch: 8 - Loss: 0.5406, Accuracy: 99.23%

Test set: Average loss: 0.5404, Accuracy: 9913/10000 (99.13%)

New best model saved with accuracy: 99.13%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.552877
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.537030
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.527474
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.570895
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.545983
Train Epoch: 9 - Loss: 0.5379, Accuracy: 99.32%

Test set: Average loss: 0.5394, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.556870
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.548177
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.541840
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.536900
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.543894
Train Epoch: 10 - Loss: 0.5357, Accuracy: 99.36%

Test set: Average loss: 0.5390, Accuracy: 9899/10000 (98.99%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.522607
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.540215
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.526040
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.532912
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.532404
Train Epoch: 11 - Loss: 0.5343, Accuracy: 99.39%

Test set: Average loss: 0.5383, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.525203
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.524311
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.517574
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.530096
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.540926
Train Epoch: 12 - Loss: 0.5334, Accuracy: 99.44%

Test set: Average loss: 0.5434, Accuracy: 9885/10000 (98.85%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.540690
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.532373
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.528352
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.519475
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.523475
Train Epoch: 13 - Loss: 0.5319, Accuracy: 99.50%

Test set: Average loss: 0.5351, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.526099
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.525680
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.525450
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.528309
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.524652
Train Epoch: 14 - Loss: 0.5304, Accuracy: 99.51%

Test set: Average loss: 0.5342, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.520723
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.531439
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.527759
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.515939
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.520527
Train Epoch: 15 - Loss: 0.5302, Accuracy: 99.48%

Test set: Average loss: 0.5335, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.528957
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.531926
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.535755
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.516416
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.512995
Train Epoch: 16 - Loss: 0.5292, Accuracy: 99.57%

Test set: Average loss: 0.5321, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.539316
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.529001
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.527348
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.521191
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.516815
Train Epoch: 17 - Loss: 0.5284, Accuracy: 99.56%

Test set: Average loss: 0.5327, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.516331
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.533003
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.546244
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.519801
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.531244
Train Epoch: 18 - Loss: 0.5282, Accuracy: 99.56%

Test set: Average loss: 0.5312, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.529302
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.522733
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.523475
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.534422
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.542426
Train Epoch: 19 - Loss: 0.5280, Accuracy: 99.57%

Test set: Average loss: 0.5325, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.516643
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.519988
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.522826
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.530479
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.531240
Train Epoch: 20 - Loss: 0.5273, Accuracy: 99.62%

Test set: Average loss: 0.5310, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.531622
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.529619
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.522415
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.530666
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.557109
Train Epoch: 21 - Loss: 0.5271, Accuracy: 99.62%

Test set: Average loss: 0.5306, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.524639
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.520661
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.515830
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.518150
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.520592
Train Epoch: 22 - Loss: 0.5272, Accuracy: 99.60%

Test set: Average loss: 0.5306, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.518443
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.515670
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.524279
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.533045
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.528883
Train Epoch: 23 - Loss: 0.5266, Accuracy: 99.63%

Test set: Average loss: 0.5308, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.519693
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.523133
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.517664
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.525105
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.515806
Train Epoch: 24 - Loss: 0.5269, Accuracy: 99.60%

Test set: Average loss: 0.5307, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.545481
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.532519
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.516071
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.534191
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.519892
Train Epoch: 25 - Loss: 0.5268, Accuracy: 99.59%

Test set: Average loss: 0.5305, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.515195
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.532011
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.539364
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.538705
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.523395
Train Epoch: 26 - Loss: 0.5294, Accuracy: 99.52%

Test set: Average loss: 0.5431, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.522931
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.529185
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.522636
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.531384
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.547467
Train Epoch: 27 - Loss: 0.5278, Accuracy: 99.57%

Test set: Average loss: 0.5309, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.539151
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.533980
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.515905
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.523127
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.521878
Train Epoch: 28 - Loss: 0.5271, Accuracy: 99.58%

Test set: Average loss: 0.5303, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.533014
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.558177
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.513964
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.522801
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.527362
Train Epoch: 29 - Loss: 0.5262, Accuracy: 99.59%

Test set: Average loss: 0.5290, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.531006
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.522187
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.521241
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.537230
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.513069
Train Epoch: 30 - Loss: 0.5255, Accuracy: 99.61%

Test set: Average loss: 0.5293, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.518922
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.515812
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.522834
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.521127
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.522841
Train Epoch: 31 - Loss: 0.5245, Accuracy: 99.62%

Test set: Average loss: 0.5284, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.518602
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.524877
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.524391
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.514749
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.557933
Train Epoch: 32 - Loss: 0.5238, Accuracy: 99.62%

Test set: Average loss: 0.5284, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.532809
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.512659
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.512324
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.524234
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.530644
Train Epoch: 33 - Loss: 0.5232, Accuracy: 99.67%

Test set: Average loss: 0.5283, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.524082
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.515018
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.516222
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.519797
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.522359
Train Epoch: 34 - Loss: 0.5228, Accuracy: 99.66%

Test set: Average loss: 0.5268, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.523932
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.513403
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.521617
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.515706
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.516308
Train Epoch: 35 - Loss: 0.5215, Accuracy: 99.72%

Test set: Average loss: 0.5264, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.523607
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.525418
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.514728
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.518813
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.513061
Train Epoch: 36 - Loss: 0.5209, Accuracy: 99.72%

Test set: Average loss: 0.5261, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.534254
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.520674
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.515518
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.511120
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.511050
Train Epoch: 37 - Loss: 0.5211, Accuracy: 99.70%

Test set: Average loss: 0.5271, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.515544
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.512974
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.531655
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.512195
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.524393
Train Epoch: 38 - Loss: 0.5205, Accuracy: 99.72%

Test set: Average loss: 0.5253, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.511593
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.514449
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.541750
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.516438
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.515039
Train Epoch: 39 - Loss: 0.5199, Accuracy: 99.77%

Test set: Average loss: 0.5266, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.516406
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.524782
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.524129
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.528567
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.535544
Train Epoch: 40 - Loss: 0.5197, Accuracy: 99.72%

Test set: Average loss: 0.5247, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.510527
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.516891
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.522336
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.518582
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.514164
Train Epoch: 41 - Loss: 0.5193, Accuracy: 99.76%

Test set: Average loss: 0.5247, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.512833
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.519107
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.511097
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.521966
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.511476
Train Epoch: 42 - Loss: 0.5189, Accuracy: 99.78%

Test set: Average loss: 0.5248, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.514527
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.523026
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.513708
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.523020
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.517753
Train Epoch: 43 - Loss: 0.5188, Accuracy: 99.80%

Test set: Average loss: 0.5247, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.518292
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.509169
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.512995
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.521845
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.521482
Train Epoch: 44 - Loss: 0.5185, Accuracy: 99.78%

Test set: Average loss: 0.5252, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.515099
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.511159
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.518790
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.513073
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.529080
Train Epoch: 45 - Loss: 0.5184, Accuracy: 99.79%

Test set: Average loss: 0.5243, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.513218
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.514146
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.521356
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.516166
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.519163
Train Epoch: 46 - Loss: 0.5183, Accuracy: 99.80%

Test set: Average loss: 0.5248, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.509331
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.515433
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.514102
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.517981
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.522344
Train Epoch: 47 - Loss: 0.5184, Accuracy: 99.80%

Test set: Average loss: 0.5245, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.519054
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.515800
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.529906
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.514901
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.524594
Train Epoch: 48 - Loss: 0.5181, Accuracy: 99.80%

Test set: Average loss: 0.5245, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.511639
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.512657
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.513716
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.531470
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.514078
Train Epoch: 49 - Loss: 0.5178, Accuracy: 99.82%

Test set: Average loss: 0.5243, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.515151
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.518664
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.513580
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.512395
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.514782
Train Epoch: 50 - Loss: 0.5178, Accuracy: 99.80%

Test set: Average loss: 0.5242, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.515560
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.518932
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.513223
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.515553
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.516938
Train Epoch: 51 - Loss: 0.5202, Accuracy: 99.72%

Test set: Average loss: 0.5273, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.514463
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.537247
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.515402
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.519252
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.520036
Train Epoch: 52 - Loss: 0.5194, Accuracy: 99.76%

Test set: Average loss: 0.5263, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.524934
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.515934
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.517386
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.556522
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.519734
Train Epoch: 53 - Loss: 0.5195, Accuracy: 99.75%

Test set: Average loss: 0.5278, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.516393
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.517998
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.516128
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.515709
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.513073
Train Epoch: 54 - Loss: 0.5187, Accuracy: 99.75%

Test set: Average loss: 0.5369, Accuracy: 9886/10000 (98.86%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.525956
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.510932
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.520390
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.532097
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.525934
Train Epoch: 55 - Loss: 0.5180, Accuracy: 99.78%

Test set: Average loss: 0.5246, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.508823
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.514245
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.515946
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.523317
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.511058
Train Epoch: 56 - Loss: 0.5179, Accuracy: 99.78%

Test set: Average loss: 0.5237, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.523939
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.521314
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.513332
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.515052
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.523796
Train Epoch: 57 - Loss: 0.5177, Accuracy: 99.77%

Test set: Average loss: 0.5240, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.526818
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.517605
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.521925
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.516913
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.515374
Train Epoch: 58 - Loss: 0.5172, Accuracy: 99.78%

Test set: Average loss: 0.5228, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.524308
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.509721
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.512517
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.514320
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.512578
Train Epoch: 59 - Loss: 0.5165, Accuracy: 99.82%

Test set: Average loss: 0.5233, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.512653
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.511412
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.516241
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.517367
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.516402
Train Epoch: 60 - Loss: 0.5163, Accuracy: 99.82%

Test set: Average loss: 0.5224, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.511086
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.510629
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.510987
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.516696
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.518876
Train Epoch: 61 - Loss: 0.5159, Accuracy: 99.83%

Test set: Average loss: 0.5224, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.515761
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.517443
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.512710
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.510826
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.514665
Train Epoch: 62 - Loss: 0.5160, Accuracy: 99.81%

Test set: Average loss: 0.5222, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.520114
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.509570
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.512936
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.519989
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.511753
Train Epoch: 63 - Loss: 0.5154, Accuracy: 99.83%

Test set: Average loss: 0.5222, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.529279
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.510866
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.513091
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.511138
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.508739
Train Epoch: 64 - Loss: 0.5153, Accuracy: 99.83%

Test set: Average loss: 0.5221, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.519550
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.516305
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.514670
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.513781
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.511749
Train Epoch: 65 - Loss: 0.5150, Accuracy: 99.85%

Test set: Average loss: 0.5221, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.514383
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.517594
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.516774
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.509118
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.515885
Train Epoch: 66 - Loss: 0.5147, Accuracy: 99.84%

Test set: Average loss: 0.5218, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.514492
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.525685
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.514005
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.509158
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.527866
Train Epoch: 67 - Loss: 0.5148, Accuracy: 99.83%

Test set: Average loss: 0.5213, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.524273
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.523493
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.512791
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.515609
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.511286
Train Epoch: 68 - Loss: 0.5145, Accuracy: 99.86%

Test set: Average loss: 0.5214, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.517384
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.508929
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.519100
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.514685
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.521142
Train Epoch: 69 - Loss: 0.5143, Accuracy: 99.86%

Test set: Average loss: 0.5215, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.522277
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.523340
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.508816
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.512228
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.512632
Train Epoch: 70 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5215, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.515052
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.507385
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.511281
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.519741
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.512469
Train Epoch: 71 - Loss: 0.5140, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.515959
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.515171
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.508579
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512286
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.511464
Train Epoch: 72 - Loss: 0.5139, Accuracy: 99.88%

Test set: Average loss: 0.5216, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.514894
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.517478
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.509808
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.516601
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.510987
Train Epoch: 73 - Loss: 0.5137, Accuracy: 99.89%

Test set: Average loss: 0.5214, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.511689
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.510881
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.520153
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.520905
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513663
Train Epoch: 74 - Loss: 0.5140, Accuracy: 99.88%

Test set: Average loss: 0.5213, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.510857
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.509308
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.514272
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.528097
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.514111
Train Epoch: 75 - Loss: 0.5140, Accuracy: 99.88%

Test set: Average loss: 0.5214, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.513556
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.512757
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.509986
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.510976
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.509878
Train Epoch: 76 - Loss: 0.5156, Accuracy: 99.82%

Test set: Average loss: 0.5287, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.519562
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.510675
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.515573
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.515608
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.507392
Train Epoch: 77 - Loss: 0.5152, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.515293
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.513776
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.509695
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.512408
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.509044
Train Epoch: 78 - Loss: 0.5151, Accuracy: 99.83%

Test set: Average loss: 0.5232, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.515051
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.513468
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.511216
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.515456
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.515102
Train Epoch: 79 - Loss: 0.5150, Accuracy: 99.82%

Test set: Average loss: 0.5226, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.510517
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.511104
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.514547
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.508241
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.519033
Train Epoch: 80 - Loss: 0.5144, Accuracy: 99.84%

Test set: Average loss: 0.5221, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.515243
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.512058
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.514772
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.507342
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.512201
Train Epoch: 81 - Loss: 0.5145, Accuracy: 99.83%

Test set: Average loss: 0.5218, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.516679
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.515422
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.530276
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.508115
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.515284
Train Epoch: 82 - Loss: 0.5140, Accuracy: 99.88%

Test set: Average loss: 0.5207, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.520981
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512994
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.510981
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.506720
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.519118
Train Epoch: 83 - Loss: 0.5138, Accuracy: 99.85%

Test set: Average loss: 0.5213, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.511621
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.549534
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.512142
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.509414
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.511248
Train Epoch: 84 - Loss: 0.5133, Accuracy: 99.87%

Test set: Average loss: 0.5292, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.509937
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.506844
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.512144
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.512398
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.508588
Train Epoch: 85 - Loss: 0.5133, Accuracy: 99.88%

Test set: Average loss: 0.5209, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.510273
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.517736
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.510742
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.513801
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.513666
Train Epoch: 86 - Loss: 0.5130, Accuracy: 99.89%

Test set: Average loss: 0.5207, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.517033
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.512313
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.511826
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.513563
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.509153
Train Epoch: 87 - Loss: 0.5129, Accuracy: 99.89%

Test set: Average loss: 0.5217, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.511945
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.506805
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.516865
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.511133
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.526466
Train Epoch: 88 - Loss: 0.5128, Accuracy: 99.89%

Test set: Average loss: 0.5205, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.507492
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.507843
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.511734
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.520412
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.524203
Train Epoch: 89 - Loss: 0.5125, Accuracy: 99.90%

Test set: Average loss: 0.5208, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.512828
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.510053
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.522997
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.506615
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.508706
Train Epoch: 90 - Loss: 0.5123, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.510448
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.507709
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509944
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.515635
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.524129
Train Epoch: 91 - Loss: 0.5121, Accuracy: 99.92%

Test set: Average loss: 0.5206, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.508859
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.505967
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.514400
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.509169
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.514342
Train Epoch: 92 - Loss: 0.5122, Accuracy: 99.89%

Test set: Average loss: 0.5205, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.513125
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.510792
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.515196
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.514721
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.507861
Train Epoch: 93 - Loss: 0.5119, Accuracy: 99.91%

Test set: Average loss: 0.5203, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.506438
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.514753
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.511692
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.507401
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.512885
Train Epoch: 94 - Loss: 0.5117, Accuracy: 99.90%

Test set: Average loss: 0.5201, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.513250
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.511622
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.519696
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.512084
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.521141
Train Epoch: 95 - Loss: 0.5118, Accuracy: 99.91%

Test set: Average loss: 0.5199, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.515230
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508336
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.506024
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.514763
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.511864
Train Epoch: 96 - Loss: 0.5115, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.507928
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.510622
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.509764
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.509069
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.512018
Train Epoch: 97 - Loss: 0.5117, Accuracy: 99.91%

Test set: Average loss: 0.5202, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.517922
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.516287
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.515737
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.515840
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.510865
Train Epoch: 98 - Loss: 0.5115, Accuracy: 99.91%

Test set: Average loss: 0.5199, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.517168
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.513164
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.517441
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.510101
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.509395
Train Epoch: 99 - Loss: 0.5114, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.521105
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.510382
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508988
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.521613
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.516892
Train Epoch: 100 - Loss: 0.5115, Accuracy: 99.91%

Test set: Average loss: 0.5198, Accuracy: 9935/10000 (99.35%)


Training completed. Best Accuracy: 99.38%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.894065
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.732690
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.667780
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.681310
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.630019
Train Epoch: 1 - Loss: 0.7305, Accuracy: 93.95%

Test set: Average loss: 0.6034, Accuracy: 9764/10000 (97.64%)

New best model saved with accuracy: 97.64%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.618484
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.620837
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.566651
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.582444
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.576099
Train Epoch: 2 - Loss: 0.5876, Accuracy: 98.00%

Test set: Average loss: 0.5682, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.553886
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.567065
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.580958
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.612599
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.564814
Train Epoch: 3 - Loss: 0.5680, Accuracy: 98.53%

Test set: Average loss: 0.5589, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.569336
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.567211
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.549752
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.573338
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.551529
Train Epoch: 4 - Loss: 0.5578, Accuracy: 98.78%

Test set: Average loss: 0.5545, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.529879
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.559714
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.565171
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.533960
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.552636
Train Epoch: 5 - Loss: 0.5512, Accuracy: 98.94%

Test set: Average loss: 0.5476, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.547711
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.538054
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.552480
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.552279
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.541203
Train Epoch: 6 - Loss: 0.5468, Accuracy: 99.03%

Test set: Average loss: 0.5444, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.566939
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.536863
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.572489
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.547601
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.534184
Train Epoch: 7 - Loss: 0.5428, Accuracy: 99.15%

Test set: Average loss: 0.5410, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.525999
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.583972
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.541077
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.542951
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.541596
Train Epoch: 8 - Loss: 0.5402, Accuracy: 99.24%

Test set: Average loss: 0.5404, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.538004
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.535593
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.521518
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.535500
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.536325
Train Epoch: 9 - Loss: 0.5381, Accuracy: 99.29%

Test set: Average loss: 0.5379, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.532282
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.536816
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.528584
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.518370
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.520760
Train Epoch: 10 - Loss: 0.5364, Accuracy: 99.32%

Test set: Average loss: 0.5366, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.578545
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.538689
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.530777
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.526965
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.538895
Train Epoch: 11 - Loss: 0.5340, Accuracy: 99.44%

Test set: Average loss: 0.5376, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.522075
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.535165
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.527962
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.529433
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.525548
Train Epoch: 12 - Loss: 0.5328, Accuracy: 99.42%

Test set: Average loss: 0.5346, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.542414
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.514976
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.522381
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.533512
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.516432
Train Epoch: 13 - Loss: 0.5317, Accuracy: 99.43%

Test set: Average loss: 0.5365, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.534726
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.522347
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.544050
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.542281
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.522745
Train Epoch: 14 - Loss: 0.5307, Accuracy: 99.47%

Test set: Average loss: 0.5329, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.516750
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.525872
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.538388
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.529483
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.527956
Train Epoch: 15 - Loss: 0.5304, Accuracy: 99.51%

Test set: Average loss: 0.5340, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.543744
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.536104
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.524306
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.527024
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.562418
Train Epoch: 16 - Loss: 0.5290, Accuracy: 99.53%

Test set: Average loss: 0.5323, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.523223
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.526701
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.529508
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.551590
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.519020
Train Epoch: 17 - Loss: 0.5287, Accuracy: 99.54%

Test set: Average loss: 0.5317, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.539077
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.533229
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.526717
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.525524
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.532584
Train Epoch: 18 - Loss: 0.5281, Accuracy: 99.58%

Test set: Average loss: 0.5316, Accuracy: 9928/10000 (99.28%)

New best model saved with accuracy: 99.28%
Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.514439
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.531706
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.523327
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.520966
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.515783
Train Epoch: 19 - Loss: 0.5281, Accuracy: 99.56%

Test set: Average loss: 0.5311, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.511012
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.541285
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.518337
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.524389
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.531994
Train Epoch: 20 - Loss: 0.5274, Accuracy: 99.59%

Test set: Average loss: 0.5311, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.515747
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.525016
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.525456
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.544993
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.526524
Train Epoch: 21 - Loss: 0.5272, Accuracy: 99.60%

Test set: Average loss: 0.5313, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.527628
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.528646
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.526693
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.510300
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.521669
Train Epoch: 22 - Loss: 0.5270, Accuracy: 99.59%

Test set: Average loss: 0.5309, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.521892
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.520942
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.526683
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.528646
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.530575
Train Epoch: 23 - Loss: 0.5267, Accuracy: 99.59%

Test set: Average loss: 0.5308, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.538874
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.519262
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.529564
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.527001
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.517835
Train Epoch: 24 - Loss: 0.5270, Accuracy: 99.59%

Test set: Average loss: 0.5308, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.516917
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.535222
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.517538
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.534033
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.525330
Train Epoch: 25 - Loss: 0.5270, Accuracy: 99.57%

Test set: Average loss: 0.5310, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.526996
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.537233
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.522731
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.517761
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.519784
Train Epoch: 26 - Loss: 0.5293, Accuracy: 99.48%

Test set: Average loss: 0.5321, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.522465
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.517179
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.529711
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.530145
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.530341
Train Epoch: 27 - Loss: 0.5282, Accuracy: 99.53%

Test set: Average loss: 0.5329, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.530595
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.537345
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.530722
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.543973
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.545149
Train Epoch: 28 - Loss: 0.5274, Accuracy: 99.54%

Test set: Average loss: 0.5303, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.541923
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.519012
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.536925
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.522063
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.518296
Train Epoch: 29 - Loss: 0.5266, Accuracy: 99.53%

Test set: Average loss: 0.5299, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.526995
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.520130
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.534808
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.555529
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.522185
Train Epoch: 30 - Loss: 0.5257, Accuracy: 99.59%

Test set: Average loss: 0.5289, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.526954
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.523074
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.514462
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.538649
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.528475
Train Epoch: 31 - Loss: 0.5247, Accuracy: 99.59%

Test set: Average loss: 0.5303, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.522951
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.513342
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.543166
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.514800
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.537612
Train Epoch: 32 - Loss: 0.5239, Accuracy: 99.64%

Test set: Average loss: 0.5281, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.530195
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.513228
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.537501
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.523620
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.518958
Train Epoch: 33 - Loss: 0.5231, Accuracy: 99.66%

Test set: Average loss: 0.5288, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.515923
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.524362
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.528218
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.513844
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.521154
Train Epoch: 34 - Loss: 0.5229, Accuracy: 99.64%

Test set: Average loss: 0.5268, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.516285
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.514962
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.529487
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.528386
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.516504
Train Epoch: 35 - Loss: 0.5223, Accuracy: 99.66%

Test set: Average loss: 0.5275, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.524947
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.509033
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.533399
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.518163
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.522297
Train Epoch: 36 - Loss: 0.5215, Accuracy: 99.70%

Test set: Average loss: 0.5264, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.514142
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.525596
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.519030
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.513021
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.522096
Train Epoch: 37 - Loss: 0.5213, Accuracy: 99.69%

Test set: Average loss: 0.5265, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.519280
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.515557
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.514468
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.516966
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.515943
Train Epoch: 38 - Loss: 0.5205, Accuracy: 99.71%

Test set: Average loss: 0.5255, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.525039
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.514637
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.523294
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.513210
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.525638
Train Epoch: 39 - Loss: 0.5200, Accuracy: 99.72%

Test set: Average loss: 0.5256, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.525189
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.529306
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.511281
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.523313
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.518829
Train Epoch: 40 - Loss: 0.5198, Accuracy: 99.75%

Test set: Average loss: 0.5252, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.520649
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.510282
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.511865
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.518807
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.516649
Train Epoch: 41 - Loss: 0.5195, Accuracy: 99.74%

Test set: Average loss: 0.5249, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.512819
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.516552
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.531617
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.533405
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.533477
Train Epoch: 42 - Loss: 0.5193, Accuracy: 99.76%

Test set: Average loss: 0.5248, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.526230
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.521348
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.518157
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.539130
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.517096
Train Epoch: 43 - Loss: 0.5188, Accuracy: 99.76%

Test set: Average loss: 0.5242, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.517203
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.520891
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.524274
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.516247
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.511219
Train Epoch: 44 - Loss: 0.5189, Accuracy: 99.76%

Test set: Average loss: 0.5241, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.511321
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.513917
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.515277
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.519602
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.510185
Train Epoch: 45 - Loss: 0.5184, Accuracy: 99.80%

Test set: Average loss: 0.5243, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.519658
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.525322
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.522353
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.514591
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.514484
Train Epoch: 46 - Loss: 0.5183, Accuracy: 99.79%

Test set: Average loss: 0.5239, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.530038
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.516834
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.519078
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.513236
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.515300
Train Epoch: 47 - Loss: 0.5180, Accuracy: 99.81%

Test set: Average loss: 0.5241, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.517007
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.513189
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.513130
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.525802
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.514364
Train Epoch: 48 - Loss: 0.5181, Accuracy: 99.78%

Test set: Average loss: 0.5242, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.511899
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.535199
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.520514
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.518168
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.520835
Train Epoch: 49 - Loss: 0.5185, Accuracy: 99.78%

Test set: Average loss: 0.5240, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.511731
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.509419
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.514298
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.515024
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.521509
Train Epoch: 50 - Loss: 0.5183, Accuracy: 99.78%

Test set: Average loss: 0.5241, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.510874
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.514945
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.527756
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.514074
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.511702
Train Epoch: 51 - Loss: 0.5207, Accuracy: 99.69%

Test set: Average loss: 0.5267, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.514606
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.517841
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.531092
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.529229
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.513298
Train Epoch: 52 - Loss: 0.5201, Accuracy: 99.72%

Test set: Average loss: 0.5250, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.519220
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.516338
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.519847
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.517956
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.535860
Train Epoch: 53 - Loss: 0.5195, Accuracy: 99.72%

Test set: Average loss: 0.5253, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.524686
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.523676
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.525949
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.512742
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.517033
Train Epoch: 54 - Loss: 0.5191, Accuracy: 99.74%

Test set: Average loss: 0.5239, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.513693
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.521291
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.516591
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.511927
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.514198
Train Epoch: 55 - Loss: 0.5187, Accuracy: 99.74%

Test set: Average loss: 0.5239, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.509835
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.512629
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.510952
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.514793
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.524625
Train Epoch: 56 - Loss: 0.5183, Accuracy: 99.76%

Test set: Average loss: 0.5230, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.515238
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.537737
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.510983
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.529882
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.511554
Train Epoch: 57 - Loss: 0.5181, Accuracy: 99.74%

Test set: Average loss: 0.5236, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.515869
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.515709
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.518337
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.512724
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.523051
Train Epoch: 58 - Loss: 0.5175, Accuracy: 99.78%

Test set: Average loss: 0.5223, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.512719
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.521078
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.518479
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.518494
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.512421
Train Epoch: 59 - Loss: 0.5168, Accuracy: 99.82%

Test set: Average loss: 0.5229, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.507714
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.520025
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.510346
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.512843
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.533424
Train Epoch: 60 - Loss: 0.5169, Accuracy: 99.79%

Test set: Average loss: 0.5263, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.514474
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.520441
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.531253
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.511611
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.515618
Train Epoch: 61 - Loss: 0.5165, Accuracy: 99.82%

Test set: Average loss: 0.5228, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.514981
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.509705
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.515167
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.517409
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.517438
Train Epoch: 62 - Loss: 0.5161, Accuracy: 99.82%

Test set: Average loss: 0.5219, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.526218
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.509379
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.515241
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.512851
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.512585
Train Epoch: 63 - Loss: 0.5156, Accuracy: 99.85%

Test set: Average loss: 0.5213, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.508654
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.529822
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.513830
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.513082
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.524841
Train Epoch: 64 - Loss: 0.5154, Accuracy: 99.83%

Test set: Average loss: 0.5215, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.508307
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.512301
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.514007
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.513894
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.512526
Train Epoch: 65 - Loss: 0.5151, Accuracy: 99.86%

Test set: Average loss: 0.5213, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.522125
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.513230
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.515057
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.511506
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.516753
Train Epoch: 66 - Loss: 0.5151, Accuracy: 99.86%

Test set: Average loss: 0.5211, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.523590
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.515156
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.520874
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.518698
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.519682
Train Epoch: 67 - Loss: 0.5153, Accuracy: 99.84%

Test set: Average loss: 0.5212, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.514449
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.510644
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.507843
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.510067
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.517352
Train Epoch: 68 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5211, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.517635
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.509473
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.514847
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.520752
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.516463
Train Epoch: 69 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5213, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.514875
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.511807
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.516916
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.514832
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.522933
Train Epoch: 70 - Loss: 0.5145, Accuracy: 99.87%

Test set: Average loss: 0.5213, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.520423
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.509326
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.511751
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.520152
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.519063
Train Epoch: 71 - Loss: 0.5144, Accuracy: 99.88%

Test set: Average loss: 0.5211, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.508190
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.510454
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.512304
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.513789
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.511930
Train Epoch: 72 - Loss: 0.5144, Accuracy: 99.87%

Test set: Average loss: 0.5212, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.513333
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.511113
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.511111
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.511785
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.511481
Train Epoch: 73 - Loss: 0.5140, Accuracy: 99.89%

Test set: Average loss: 0.5208, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.511493
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.521579
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.511251
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.511020
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.513000
Train Epoch: 74 - Loss: 0.5143, Accuracy: 99.86%

Test set: Average loss: 0.5212, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.513697
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.510279
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.517334
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.509834
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.530605
Train Epoch: 75 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5209, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.512415
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.512110
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.512751
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.513247
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.523203
Train Epoch: 76 - Loss: 0.5161, Accuracy: 99.82%

Test set: Average loss: 0.5252, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.515467
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.534822
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.510711
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.516116
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.512219
Train Epoch: 77 - Loss: 0.5157, Accuracy: 99.83%

Test set: Average loss: 0.5249, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.508887
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.520769
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.512306
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.517907
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.511767
Train Epoch: 78 - Loss: 0.5150, Accuracy: 99.84%

Test set: Average loss: 0.5217, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.511455
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.514840
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.511964
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.531600
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.513525
Train Epoch: 79 - Loss: 0.5149, Accuracy: 99.83%

Test set: Average loss: 0.5207, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.510023
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.510865
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.511126
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.516092
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.515593
Train Epoch: 80 - Loss: 0.5150, Accuracy: 99.84%

Test set: Average loss: 0.5215, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.510059
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.507218
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.513328
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.509845
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.511512
Train Epoch: 81 - Loss: 0.5149, Accuracy: 99.83%

Test set: Average loss: 0.5213, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.508582
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.513215
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.518412
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.506242
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.516902
Train Epoch: 82 - Loss: 0.5140, Accuracy: 99.88%

Test set: Average loss: 0.5222, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.509202
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.517042
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.516888
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.509100
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.511903
Train Epoch: 83 - Loss: 0.5142, Accuracy: 99.86%

Test set: Average loss: 0.5198, Accuracy: 9951/10000 (99.51%)

New best model saved with accuracy: 99.51%
Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.531551
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.519820
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.516079
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.512248
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.508665
Train Epoch: 84 - Loss: 0.5140, Accuracy: 99.87%

Test set: Average loss: 0.5207, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.513425
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.517690
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.509082
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.514611
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.520217
Train Epoch: 85 - Loss: 0.5137, Accuracy: 99.87%

Test set: Average loss: 0.5243, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.515053
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.522129
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.513311
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.511853
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.513447
Train Epoch: 86 - Loss: 0.5133, Accuracy: 99.90%

Test set: Average loss: 0.5200, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.513123
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.514263
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.513071
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.507849
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.510435
Train Epoch: 87 - Loss: 0.5132, Accuracy: 99.89%

Test set: Average loss: 0.5200, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.509128
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.509456
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.512579
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.509222
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.516689
Train Epoch: 88 - Loss: 0.5129, Accuracy: 99.89%

Test set: Average loss: 0.5196, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.507418
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.505737
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.511498
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.509139
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.506180
Train Epoch: 89 - Loss: 0.5128, Accuracy: 99.89%

Test set: Average loss: 0.5203, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.517670
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513943
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.509422
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.509652
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.507409
Train Epoch: 90 - Loss: 0.5126, Accuracy: 99.90%

Test set: Average loss: 0.5195, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.511452
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.513768
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509970
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.510595
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.507884
Train Epoch: 91 - Loss: 0.5123, Accuracy: 99.91%

Test set: Average loss: 0.5195, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.512742
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510630
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.511604
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.516491
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.512337
Train Epoch: 92 - Loss: 0.5123, Accuracy: 99.90%

Test set: Average loss: 0.5195, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.513343
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.507560
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.509749
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.510693
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.517169
Train Epoch: 93 - Loss: 0.5121, Accuracy: 99.93%

Test set: Average loss: 0.5194, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.513758
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.507111
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.507926
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.512441
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.510282
Train Epoch: 94 - Loss: 0.5121, Accuracy: 99.90%

Test set: Average loss: 0.5196, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.507542
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.515016
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.508512
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.508800
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.508115
Train Epoch: 95 - Loss: 0.5120, Accuracy: 99.91%

Test set: Average loss: 0.5195, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.510055
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508035
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.547038
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.508682
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.512945
Train Epoch: 96 - Loss: 0.5120, Accuracy: 99.91%

Test set: Average loss: 0.5193, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.520703
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.507580
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.507819
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.522772
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.505804
Train Epoch: 97 - Loss: 0.5119, Accuracy: 99.92%

Test set: Average loss: 0.5192, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.510195
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.512473
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.513000
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508912
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.513597
Train Epoch: 98 - Loss: 0.5116, Accuracy: 99.93%

Test set: Average loss: 0.5192, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.510455
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.508291
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.516231
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.515850
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.533331
Train Epoch: 99 - Loss: 0.5117, Accuracy: 99.92%

Test set: Average loss: 0.5194, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.508759
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.506368
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.513066
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.515384
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.517138
Train Epoch: 100 - Loss: 0.5117, Accuracy: 99.93%

Test set: Average loss: 0.5195, Accuracy: 9948/10000 (99.48%)


Training completed. Best Accuracy: 99.52%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.789487
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.700735
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.656852
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.579065
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.583217
Train Epoch: 1 - Loss: 0.7262, Accuracy: 94.19%

Test set: Average loss: 0.6054, Accuracy: 9754/10000 (97.54%)

New best model saved with accuracy: 97.54%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.606055
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.572735
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.588836
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.571453
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.592972
Train Epoch: 2 - Loss: 0.5899, Accuracy: 98.19%

Test set: Average loss: 0.5730, Accuracy: 9839/10000 (98.39%)

New best model saved with accuracy: 98.39%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.566538
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.579584
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.566614
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.551694
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.578386
Train Epoch: 3 - Loss: 0.5702, Accuracy: 98.61%

Test set: Average loss: 0.5619, Accuracy: 9868/10000 (98.68%)

New best model saved with accuracy: 98.68%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.598120
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.577898
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.546327
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.540056
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.559997
Train Epoch: 4 - Loss: 0.5599, Accuracy: 98.83%

Test set: Average loss: 0.5538, Accuracy: 9882/10000 (98.82%)

New best model saved with accuracy: 98.82%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.624005
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.541858
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.550428
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.535846
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.537263
Train Epoch: 5 - Loss: 0.5531, Accuracy: 99.03%

Test set: Average loss: 0.5488, Accuracy: 9888/10000 (98.88%)

New best model saved with accuracy: 98.88%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.539463
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.550218
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.552067
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.574358
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.568341
Train Epoch: 6 - Loss: 0.5484, Accuracy: 99.11%

Test set: Average loss: 0.5453, Accuracy: 9905/10000 (99.05%)

New best model saved with accuracy: 99.05%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.539708
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.535220
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.554772
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.536357
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.538852
Train Epoch: 7 - Loss: 0.5445, Accuracy: 99.21%

Test set: Average loss: 0.5436, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.525285
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.540496
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.529213
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.540065
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.520358
Train Epoch: 8 - Loss: 0.5421, Accuracy: 99.25%

Test set: Average loss: 0.5417, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.531915
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.546400
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.541362
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.552230
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.541944
Train Epoch: 9 - Loss: 0.5397, Accuracy: 99.32%

Test set: Average loss: 0.5389, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.550390
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.532447
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.530073
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.530031
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.531143
Train Epoch: 10 - Loss: 0.5372, Accuracy: 99.38%

Test set: Average loss: 0.5386, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.551264
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.560322
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.532428
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.544659
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.524740
Train Epoch: 11 - Loss: 0.5352, Accuracy: 99.44%

Test set: Average loss: 0.5379, Accuracy: 9911/10000 (99.11%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.532039
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.529895
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.525673
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.524217
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.525868
Train Epoch: 12 - Loss: 0.5341, Accuracy: 99.44%

Test set: Average loss: 0.5369, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.524838
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.531722
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.531845
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.522396
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.524756
Train Epoch: 13 - Loss: 0.5326, Accuracy: 99.50%

Test set: Average loss: 0.5348, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.521159
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.528134
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.522331
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.570262
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.524675
Train Epoch: 14 - Loss: 0.5323, Accuracy: 99.51%

Test set: Average loss: 0.5347, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.551433
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.530170
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.515521
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.524335
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.525249
Train Epoch: 15 - Loss: 0.5314, Accuracy: 99.50%

Test set: Average loss: 0.5341, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.528077
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.519783
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.529833
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.536883
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.520290
Train Epoch: 16 - Loss: 0.5306, Accuracy: 99.52%

Test set: Average loss: 0.5338, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.538110
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.524742
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.547422
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.532154
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.534686
Train Epoch: 17 - Loss: 0.5298, Accuracy: 99.56%

Test set: Average loss: 0.5329, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.523468
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.525685
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.536373
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.530858
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.519231
Train Epoch: 18 - Loss: 0.5294, Accuracy: 99.57%

Test set: Average loss: 0.5330, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.538356
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.526339
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.529862
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.526659
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.522969
Train Epoch: 19 - Loss: 0.5291, Accuracy: 99.60%

Test set: Average loss: 0.5325, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.522804
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.529560
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.525306
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.540571
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.525441
Train Epoch: 20 - Loss: 0.5284, Accuracy: 99.61%

Test set: Average loss: 0.5325, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.553001
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.534876
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.520922
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.541352
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.524019
Train Epoch: 21 - Loss: 0.5284, Accuracy: 99.60%

Test set: Average loss: 0.5322, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.532887
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.527215
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.524809
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.536975
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.524071
Train Epoch: 22 - Loss: 0.5280, Accuracy: 99.62%

Test set: Average loss: 0.5324, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.518026
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.532717
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.528623
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.524172
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.536830
Train Epoch: 23 - Loss: 0.5282, Accuracy: 99.60%

Test set: Average loss: 0.5320, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.517752
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.530853
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.532282
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.525623
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.516088
Train Epoch: 24 - Loss: 0.5280, Accuracy: 99.62%

Test set: Average loss: 0.5322, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.534643
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.518707
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.518807
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.548140
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.524900
Train Epoch: 25 - Loss: 0.5275, Accuracy: 99.64%

Test set: Average loss: 0.5321, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.523959
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.531250
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.558840
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.545887
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.517110
Train Epoch: 26 - Loss: 0.5302, Accuracy: 99.55%

Test set: Average loss: 0.5330, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.519602
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.534171
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.534015
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.535114
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.528486
Train Epoch: 27 - Loss: 0.5292, Accuracy: 99.53%

Test set: Average loss: 0.5341, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.522641
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.536857
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.536675
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.527562
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.529494
Train Epoch: 28 - Loss: 0.5284, Accuracy: 99.56%

Test set: Average loss: 0.5318, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.525923
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.513105
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.536546
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.540478
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.524639
Train Epoch: 29 - Loss: 0.5270, Accuracy: 99.62%

Test set: Average loss: 0.5300, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.520031
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.522416
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.525416
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.529074
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.519609
Train Epoch: 30 - Loss: 0.5256, Accuracy: 99.65%

Test set: Average loss: 0.5304, Accuracy: 9928/10000 (99.28%)

New best model saved with accuracy: 99.28%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.537926
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.525298
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.515338
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.530648
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.518277
Train Epoch: 31 - Loss: 0.5251, Accuracy: 99.64%

Test set: Average loss: 0.5308, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.518772
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.526641
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.515457
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.515000
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.523999
Train Epoch: 32 - Loss: 0.5246, Accuracy: 99.67%

Test set: Average loss: 0.5291, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.536744
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.524700
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.524414
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.521125
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.527614
Train Epoch: 33 - Loss: 0.5240, Accuracy: 99.66%

Test set: Average loss: 0.5286, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.520633
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.525292
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.514428
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.525011
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.528599
Train Epoch: 34 - Loss: 0.5229, Accuracy: 99.69%

Test set: Average loss: 0.5276, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.518513
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.517766
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.515381
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.524959
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.509858
Train Epoch: 35 - Loss: 0.5225, Accuracy: 99.71%

Test set: Average loss: 0.5274, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.514570
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.539889
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.518531
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.529240
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.527807
Train Epoch: 36 - Loss: 0.5218, Accuracy: 99.75%

Test set: Average loss: 0.5275, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.517829
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.516056
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.513195
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.512059
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.520125
Train Epoch: 37 - Loss: 0.5216, Accuracy: 99.72%

Test set: Average loss: 0.5266, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.515330
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.523908
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.514724
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.514498
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.515483
Train Epoch: 38 - Loss: 0.5210, Accuracy: 99.73%

Test set: Average loss: 0.5268, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.535357
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.528214
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.523828
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.515674
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.525833
Train Epoch: 39 - Loss: 0.5205, Accuracy: 99.74%

Test set: Average loss: 0.5261, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.521581
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.516708
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.516992
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.521534
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.526236
Train Epoch: 40 - Loss: 0.5202, Accuracy: 99.77%

Test set: Average loss: 0.5259, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.514148
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.531886
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.517569
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.515878
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.525350
Train Epoch: 41 - Loss: 0.5199, Accuracy: 99.75%

Test set: Average loss: 0.5256, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.511090
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.545866
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.519027
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.535211
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.516625
Train Epoch: 42 - Loss: 0.5197, Accuracy: 99.77%

Test set: Average loss: 0.5259, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.515886
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.513251
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.524048
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.521383
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.532466
Train Epoch: 43 - Loss: 0.5194, Accuracy: 99.77%

Test set: Average loss: 0.5256, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.508732
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.521901
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.521011
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.516250
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.515694
Train Epoch: 44 - Loss: 0.5193, Accuracy: 99.77%

Test set: Average loss: 0.5254, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.513837
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.516703
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.514146
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.520839
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.513980
Train Epoch: 45 - Loss: 0.5190, Accuracy: 99.78%

Test set: Average loss: 0.5254, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.517255
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.521078
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.517869
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.521727
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.517145
Train Epoch: 46 - Loss: 0.5191, Accuracy: 99.77%

Test set: Average loss: 0.5251, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.511470
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.536269
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.520080
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.518223
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.516259
Train Epoch: 47 - Loss: 0.5187, Accuracy: 99.81%

Test set: Average loss: 0.5253, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.512200
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.512379
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.542172
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.516112
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.520370
Train Epoch: 48 - Loss: 0.5189, Accuracy: 99.79%

Test set: Average loss: 0.5252, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.522133
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.518132
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.514801
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.526626
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.517088
Train Epoch: 49 - Loss: 0.5186, Accuracy: 99.81%

Test set: Average loss: 0.5253, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.531077
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.510363
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.520093
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.518822
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.514418
Train Epoch: 50 - Loss: 0.5184, Accuracy: 99.81%

Test set: Average loss: 0.5253, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.509498
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.514150
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.515089
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.518187
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.517790
Train Epoch: 51 - Loss: 0.5204, Accuracy: 99.74%

Test set: Average loss: 0.5258, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.513463
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.521241
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.522738
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.526005
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.517129
Train Epoch: 52 - Loss: 0.5201, Accuracy: 99.75%

Test set: Average loss: 0.5254, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.547476
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.510677
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.512989
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.513649
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.519262
Train Epoch: 53 - Loss: 0.5195, Accuracy: 99.75%

Test set: Average loss: 0.5277, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.530012
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.523266
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.517510
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.520205
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.516883
Train Epoch: 54 - Loss: 0.5190, Accuracy: 99.76%

Test set: Average loss: 0.5246, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.518530
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.520627
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.513387
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.514776
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.517372
Train Epoch: 55 - Loss: 0.5189, Accuracy: 99.77%

Test set: Average loss: 0.5250, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.523705
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.514933
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514654
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.513753
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.527980
Train Epoch: 56 - Loss: 0.5185, Accuracy: 99.78%

Test set: Average loss: 0.5251, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.520151
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.537697
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.517723
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.517735
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.515374
Train Epoch: 57 - Loss: 0.5182, Accuracy: 99.80%

Test set: Average loss: 0.5244, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.513915
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.509145
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.532335
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.518490
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.514314
Train Epoch: 58 - Loss: 0.5177, Accuracy: 99.81%

Test set: Average loss: 0.5240, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.521323
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.520336
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.517057
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.516463
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.516454
Train Epoch: 59 - Loss: 0.5170, Accuracy: 99.81%

Test set: Average loss: 0.5241, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.522041
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.513734
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.516744
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.517161
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.518178
Train Epoch: 60 - Loss: 0.5167, Accuracy: 99.83%

Test set: Average loss: 0.5234, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.519210
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.513736
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.520308
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.521516
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.518885
Train Epoch: 61 - Loss: 0.5166, Accuracy: 99.84%

Test set: Average loss: 0.5235, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.510866
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.511516
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.525482
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.511847
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.509914
Train Epoch: 62 - Loss: 0.5164, Accuracy: 99.81%

Test set: Average loss: 0.5261, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.514527
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.530460
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.516012
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.511896
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.511807
Train Epoch: 63 - Loss: 0.5160, Accuracy: 99.83%

Test set: Average loss: 0.5228, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.521038
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.511091
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.509560
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.513384
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.515647
Train Epoch: 64 - Loss: 0.5159, Accuracy: 99.83%

Test set: Average loss: 0.5228, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.509631
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.512946
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.510466
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.513361
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.519393
Train Epoch: 65 - Loss: 0.5156, Accuracy: 99.85%

Test set: Average loss: 0.5227, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.516588
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.520877
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.514098
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.524410
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.508942
Train Epoch: 66 - Loss: 0.5151, Accuracy: 99.87%

Test set: Average loss: 0.5222, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.509547
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.511540
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.512017
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.511439
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.517236
Train Epoch: 67 - Loss: 0.5150, Accuracy: 99.86%

Test set: Average loss: 0.5222, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.512764
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.508035
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.506953
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.507171
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.513698
Train Epoch: 68 - Loss: 0.5149, Accuracy: 99.87%

Test set: Average loss: 0.5222, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.512701
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.514125
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.512102
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.517207
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.510827
Train Epoch: 69 - Loss: 0.5148, Accuracy: 99.86%

Test set: Average loss: 0.5223, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.518638
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.518869
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.513061
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.512864
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.518185
Train Epoch: 70 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5222, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.512521
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.512070
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.516936
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.509952
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.509705
Train Epoch: 71 - Loss: 0.5146, Accuracy: 99.87%

Test set: Average loss: 0.5222, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.509176
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.512366
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.519234
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.519198
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.509895
Train Epoch: 72 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5221, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.509819
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.510277
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.510072
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.520867
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.528400
Train Epoch: 73 - Loss: 0.5144, Accuracy: 99.88%

Test set: Average loss: 0.5221, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.522454
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.517544
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.507247
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.519589
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.512516
Train Epoch: 74 - Loss: 0.5147, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.515851
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.513161
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.514654
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.514931
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.513178
Train Epoch: 75 - Loss: 0.5145, Accuracy: 99.87%

Test set: Average loss: 0.5222, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.509120
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.513289
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.517102
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.514884
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.531242
Train Epoch: 76 - Loss: 0.5159, Accuracy: 99.83%

Test set: Average loss: 0.5229, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.508113
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.521068
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.521378
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.511694
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.513497
Train Epoch: 77 - Loss: 0.5156, Accuracy: 99.83%

Test set: Average loss: 0.5236, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.524096
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.516347
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.516318
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.515466
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.516226
Train Epoch: 78 - Loss: 0.5155, Accuracy: 99.84%

Test set: Average loss: 0.5230, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.508913
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.525792
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.519191
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.512590
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.507831
Train Epoch: 79 - Loss: 0.5151, Accuracy: 99.85%

Test set: Average loss: 0.5223, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.513337
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.514818
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.513264
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.509438
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.513778
Train Epoch: 80 - Loss: 0.5148, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.510866
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.514915
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.510199
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.512546
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.520758
Train Epoch: 81 - Loss: 0.5147, Accuracy: 99.86%

Test set: Average loss: 0.5222, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.508422
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.507256
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.511914
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.511556
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.513450
Train Epoch: 82 - Loss: 0.5143, Accuracy: 99.86%

Test set: Average loss: 0.5234, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.511446
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.509606
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.513995
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.511810
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.521170
Train Epoch: 83 - Loss: 0.5141, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.514658
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.511937
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.512229
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.506837
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.518915
Train Epoch: 84 - Loss: 0.5139, Accuracy: 99.88%

Test set: Average loss: 0.5217, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.512703
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.508183
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.507204
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.514576
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.506805
Train Epoch: 85 - Loss: 0.5134, Accuracy: 99.90%

Test set: Average loss: 0.5213, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.507465
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.509650
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.516704
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.511179
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.511741
Train Epoch: 86 - Loss: 0.5133, Accuracy: 99.88%

Test set: Average loss: 0.5213, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.519208
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.516005
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.511858
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.508798
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.514114
Train Epoch: 87 - Loss: 0.5132, Accuracy: 99.88%

Test set: Average loss: 0.5212, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.511476
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.514374
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.511850
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.512147
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.511006
Train Epoch: 88 - Loss: 0.5130, Accuracy: 99.89%

Test set: Average loss: 0.5207, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.514182
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.532553
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.510630
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.511770
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.508224
Train Epoch: 89 - Loss: 0.5128, Accuracy: 99.90%

Test set: Average loss: 0.5214, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.510591
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.506907
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.511412
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.519788
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.507633
Train Epoch: 90 - Loss: 0.5126, Accuracy: 99.89%

Test set: Average loss: 0.5205, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.512830
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.511547
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.510819
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.510230
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.516630
Train Epoch: 91 - Loss: 0.5125, Accuracy: 99.91%

Test set: Average loss: 0.5206, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.507481
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.516161
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.510447
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.508422
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.507391
Train Epoch: 92 - Loss: 0.5121, Accuracy: 99.90%

Test set: Average loss: 0.5209, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.510809
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.514252
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.509591
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.511955
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.515423
Train Epoch: 93 - Loss: 0.5123, Accuracy: 99.91%

Test set: Average loss: 0.5204, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.513586
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.508285
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.508489
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.508792
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.508864
Train Epoch: 94 - Loss: 0.5120, Accuracy: 99.91%

Test set: Average loss: 0.5205, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.508316
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.511924
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.519647
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.510806
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.515873
Train Epoch: 95 - Loss: 0.5121, Accuracy: 99.91%

Test set: Average loss: 0.5204, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.522669
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.509909
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.512568
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.506518
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.511776
Train Epoch: 96 - Loss: 0.5121, Accuracy: 99.92%

Test set: Average loss: 0.5205, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.512133
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.507748
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.508862
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.524961
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.509701
Train Epoch: 97 - Loss: 0.5119, Accuracy: 99.91%

Test set: Average loss: 0.5204, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.510517
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.510112
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.510690
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511272
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.511864
Train Epoch: 98 - Loss: 0.5117, Accuracy: 99.92%

Test set: Average loss: 0.5205, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.509358
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.510408
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.510758
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.508567
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.512033
Train Epoch: 99 - Loss: 0.5117, Accuracy: 99.93%

Test set: Average loss: 0.5204, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.509426
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.520034
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.507997
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.514223
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.509659
Train Epoch: 100 - Loss: 0.5119, Accuracy: 99.91%

Test set: Average loss: 0.5203, Accuracy: 9939/10000 (99.39%)


Training completed. Best Accuracy: 99.41%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.710530
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.678750
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.716385
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.629142
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.607539
Train Epoch: 1 - Loss: 0.7164, Accuracy: 94.43%

Test set: Average loss: 0.5964, Accuracy: 9784/10000 (97.84%)

New best model saved with accuracy: 97.84%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.573133
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.592380
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.587436
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.574633
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.549538
Train Epoch: 2 - Loss: 0.5875, Accuracy: 98.14%

Test set: Average loss: 0.5759, Accuracy: 9833/10000 (98.33%)

New best model saved with accuracy: 98.33%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.580057
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.566208
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.573900
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.559508
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.537607
Train Epoch: 3 - Loss: 0.5671, Accuracy: 98.67%

Test set: Average loss: 0.5625, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.587566
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.560126
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.552204
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.552460
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.557422
Train Epoch: 4 - Loss: 0.5574, Accuracy: 98.89%

Test set: Average loss: 0.5569, Accuracy: 9858/10000 (98.58%)

New best model saved with accuracy: 98.58%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.545190
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.543015
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.579861
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.544632
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.530385
Train Epoch: 5 - Loss: 0.5507, Accuracy: 99.01%

Test set: Average loss: 0.5475, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.536848
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.532361
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.572355
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.543223
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.549697
Train Epoch: 6 - Loss: 0.5462, Accuracy: 99.09%

Test set: Average loss: 0.5457, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.529018
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.545622
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.531933
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.531143
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.547720
Train Epoch: 7 - Loss: 0.5425, Accuracy: 99.20%

Test set: Average loss: 0.5437, Accuracy: 9898/10000 (98.98%)

New best model saved with accuracy: 98.98%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.534384
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.525852
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.521337
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.606136
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.537378
Train Epoch: 8 - Loss: 0.5402, Accuracy: 99.24%

Test set: Average loss: 0.5417, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.530883
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.530839
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.526933
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.548073
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.532420
Train Epoch: 9 - Loss: 0.5377, Accuracy: 99.31%

Test set: Average loss: 0.5392, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.531807
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.531735
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.522331
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.522230
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.531381
Train Epoch: 10 - Loss: 0.5359, Accuracy: 99.32%

Test set: Average loss: 0.5417, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.528376
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.534590
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.559019
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.533467
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.523806
Train Epoch: 11 - Loss: 0.5344, Accuracy: 99.41%

Test set: Average loss: 0.5355, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.553732
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.556481
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.525446
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.526259
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.558553
Train Epoch: 12 - Loss: 0.5328, Accuracy: 99.39%

Test set: Average loss: 0.5379, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.525904
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.561535
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.528460
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.522050
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.524663
Train Epoch: 13 - Loss: 0.5323, Accuracy: 99.42%

Test set: Average loss: 0.5341, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.533491
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.518957
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.538975
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.543198
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.544448
Train Epoch: 14 - Loss: 0.5308, Accuracy: 99.50%

Test set: Average loss: 0.5330, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.522333
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.530067
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.516066
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.520071
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.532971
Train Epoch: 15 - Loss: 0.5301, Accuracy: 99.52%

Test set: Average loss: 0.5321, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.540489
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.520738
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.532644
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.526937
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.528892
Train Epoch: 16 - Loss: 0.5295, Accuracy: 99.52%

Test set: Average loss: 0.5321, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.522040
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.536590
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.524366
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.540692
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.538470
Train Epoch: 17 - Loss: 0.5287, Accuracy: 99.53%

Test set: Average loss: 0.5319, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.520364
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.548476
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.524521
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.529179
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.535816
Train Epoch: 18 - Loss: 0.5283, Accuracy: 99.58%

Test set: Average loss: 0.5314, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.545135
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.525034
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.523880
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.531900
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.539571
Train Epoch: 19 - Loss: 0.5280, Accuracy: 99.55%

Test set: Average loss: 0.5310, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.524703
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.520991
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.530829
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.515711
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.530637
Train Epoch: 20 - Loss: 0.5274, Accuracy: 99.59%

Test set: Average loss: 0.5310, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.538969
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.525176
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.526040
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.534367
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.523540
Train Epoch: 21 - Loss: 0.5272, Accuracy: 99.55%

Test set: Average loss: 0.5313, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.532811
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.528537
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.530988
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.531039
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.525620
Train Epoch: 22 - Loss: 0.5269, Accuracy: 99.60%

Test set: Average loss: 0.5309, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.526742
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.518808
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.519876
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.522359
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.520194
Train Epoch: 23 - Loss: 0.5272, Accuracy: 99.58%

Test set: Average loss: 0.5309, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.513896
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.516316
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.517384
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.521368
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.515837
Train Epoch: 24 - Loss: 0.5269, Accuracy: 99.60%

Test set: Average loss: 0.5307, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.539396
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.517958
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.534031
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.522075
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.524350
Train Epoch: 25 - Loss: 0.5267, Accuracy: 99.60%

Test set: Average loss: 0.5308, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.541905
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.521411
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.532135
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.528584
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.524229
Train Epoch: 26 - Loss: 0.5294, Accuracy: 99.51%

Test set: Average loss: 0.5329, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.527078
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.516762
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.519148
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.518324
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.546555
Train Epoch: 27 - Loss: 0.5283, Accuracy: 99.54%

Test set: Average loss: 0.5323, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.531480
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.530058
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.534494
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.517174
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.532396
Train Epoch: 28 - Loss: 0.5274, Accuracy: 99.53%

Test set: Average loss: 0.5312, Accuracy: 9928/10000 (99.28%)

New best model saved with accuracy: 99.28%
Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.516696
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.519517
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.517121
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.527643
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.521789
Train Epoch: 29 - Loss: 0.5261, Accuracy: 99.57%

Test set: Average loss: 0.5325, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.513815
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.548663
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.516722
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.530754
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.522274
Train Epoch: 30 - Loss: 0.5253, Accuracy: 99.58%

Test set: Average loss: 0.5305, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.519897
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.521968
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.519775
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.513354
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.515944
Train Epoch: 31 - Loss: 0.5248, Accuracy: 99.61%

Test set: Average loss: 0.5288, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.520094
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.521489
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.522163
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.523797
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.516861
Train Epoch: 32 - Loss: 0.5237, Accuracy: 99.64%

Test set: Average loss: 0.5280, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.528566
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.522163
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.509823
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.543630
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.529533
Train Epoch: 33 - Loss: 0.5229, Accuracy: 99.66%

Test set: Average loss: 0.5272, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.517421
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.526604
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.513731
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.514107
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.513518
Train Epoch: 34 - Loss: 0.5224, Accuracy: 99.67%

Test set: Average loss: 0.5290, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.521788
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.516003
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.515189
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.514582
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.518307
Train Epoch: 35 - Loss: 0.5217, Accuracy: 99.67%

Test set: Average loss: 0.5270, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.514839
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.524228
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.520526
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.518045
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.524236
Train Epoch: 36 - Loss: 0.5210, Accuracy: 99.71%

Test set: Average loss: 0.5281, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.523700
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.516688
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.524723
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.518449
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.515331
Train Epoch: 37 - Loss: 0.5208, Accuracy: 99.72%

Test set: Average loss: 0.5266, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.517310
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.517094
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.514757
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.520525
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.521012
Train Epoch: 38 - Loss: 0.5200, Accuracy: 99.74%

Test set: Average loss: 0.5258, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.523088
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.515956
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.519042
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.514711
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.515486
Train Epoch: 39 - Loss: 0.5197, Accuracy: 99.75%

Test set: Average loss: 0.5257, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.517260
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.519181
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.532696
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.519576
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.516169
Train Epoch: 40 - Loss: 0.5198, Accuracy: 99.74%

Test set: Average loss: 0.5250, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.512518
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.530120
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.519688
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.514145
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.527005
Train Epoch: 41 - Loss: 0.5189, Accuracy: 99.76%

Test set: Average loss: 0.5254, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.542827
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.517509
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.514475
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.527675
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.514514
Train Epoch: 42 - Loss: 0.5187, Accuracy: 99.77%

Test set: Average loss: 0.5248, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.511750
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.519998
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.514347
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.516519
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.511878
Train Epoch: 43 - Loss: 0.5188, Accuracy: 99.76%

Test set: Average loss: 0.5249, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.515194
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.514739
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.514801
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.514158
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.516305
Train Epoch: 44 - Loss: 0.5184, Accuracy: 99.77%

Test set: Average loss: 0.5247, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.531142
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.515830
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.507658
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.515594
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.531082
Train Epoch: 45 - Loss: 0.5185, Accuracy: 99.77%

Test set: Average loss: 0.5247, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.520054
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.514340
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.511076
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.515361
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.515708
Train Epoch: 46 - Loss: 0.5180, Accuracy: 99.82%

Test set: Average loss: 0.5245, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.512606
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.517740
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.512333
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.515234
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.545568
Train Epoch: 47 - Loss: 0.5181, Accuracy: 99.80%

Test set: Average loss: 0.5245, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.513170
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.512684
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.513765
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.517641
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.526810
Train Epoch: 48 - Loss: 0.5178, Accuracy: 99.79%

Test set: Average loss: 0.5247, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.517728
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.513671
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.507495
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.512779
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.536272
Train Epoch: 49 - Loss: 0.5179, Accuracy: 99.81%

Test set: Average loss: 0.5244, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.515593
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.510487
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.509346
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.519541
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.531158
Train Epoch: 50 - Loss: 0.5178, Accuracy: 99.80%

Test set: Average loss: 0.5246, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.516124
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.509954
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.510501
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.519514
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.519103
Train Epoch: 51 - Loss: 0.5201, Accuracy: 99.71%

Test set: Average loss: 0.5260, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.517170
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.519065
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.517748
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.513794
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.525990
Train Epoch: 52 - Loss: 0.5196, Accuracy: 99.72%

Test set: Average loss: 0.5269, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.518102
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.514460
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.516681
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.530205
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.518049
Train Epoch: 53 - Loss: 0.5192, Accuracy: 99.72%

Test set: Average loss: 0.5297, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.514430
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.515618
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.511202
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.514729
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.515258
Train Epoch: 54 - Loss: 0.5183, Accuracy: 99.76%

Test set: Average loss: 0.5253, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.511862
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.527403
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.528360
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.515119
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.515413
Train Epoch: 55 - Loss: 0.5182, Accuracy: 99.78%

Test set: Average loss: 0.5261, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.517030
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.517435
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.521521
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.513339
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.512235
Train Epoch: 56 - Loss: 0.5180, Accuracy: 99.78%

Test set: Average loss: 0.5240, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.511852
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.524116
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.520770
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.510196
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.521155
Train Epoch: 57 - Loss: 0.5174, Accuracy: 99.77%

Test set: Average loss: 0.5244, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.529840
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.514885
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.514351
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.513931
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.516143
Train Epoch: 58 - Loss: 0.5168, Accuracy: 99.81%

Test set: Average loss: 0.5233, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.514019
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.514420
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.510657
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.522553
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.518273
Train Epoch: 59 - Loss: 0.5166, Accuracy: 99.80%

Test set: Average loss: 0.5237, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.518503
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.513854
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.511119
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.512863
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.514899
Train Epoch: 60 - Loss: 0.5161, Accuracy: 99.81%

Test set: Average loss: 0.5232, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.512601
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.513432
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.516980
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.514077
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.513354
Train Epoch: 61 - Loss: 0.5157, Accuracy: 99.84%

Test set: Average loss: 0.5230, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.515417
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.519547
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.517324
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.509623
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.519737
Train Epoch: 62 - Loss: 0.5154, Accuracy: 99.86%

Test set: Average loss: 0.5227, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.507585
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.510438
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.524616
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.517028
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.513174
Train Epoch: 63 - Loss: 0.5151, Accuracy: 99.82%

Test set: Average loss: 0.5230, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.528488
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.508133
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.512877
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.511439
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.514104
Train Epoch: 64 - Loss: 0.5149, Accuracy: 99.86%

Test set: Average loss: 0.5222, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.514129
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.508882
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.517910
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.511437
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.520747
Train Epoch: 65 - Loss: 0.5149, Accuracy: 99.85%

Test set: Average loss: 0.5222, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.513806
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.510931
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.511629
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.516121
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.512084
Train Epoch: 66 - Loss: 0.5145, Accuracy: 99.86%

Test set: Average loss: 0.5229, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.513505
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.512786
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.507196
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.519541
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.509156
Train Epoch: 67 - Loss: 0.5143, Accuracy: 99.86%

Test set: Average loss: 0.5222, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.510604
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.512372
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.515838
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.522998
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.511879
Train Epoch: 68 - Loss: 0.5142, Accuracy: 99.87%

Test set: Average loss: 0.5227, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.510068
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.530975
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.511625
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.512848
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.511193
Train Epoch: 69 - Loss: 0.5142, Accuracy: 99.89%

Test set: Average loss: 0.5222, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.513384
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.513680
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.510936
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.524865
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.507837
Train Epoch: 70 - Loss: 0.5139, Accuracy: 99.88%

Test set: Average loss: 0.5220, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.508164
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.511826
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.508932
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.509064
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.521559
Train Epoch: 71 - Loss: 0.5138, Accuracy: 99.88%

Test set: Average loss: 0.5223, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.515729
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.510050
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.519839
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512941
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.513314
Train Epoch: 72 - Loss: 0.5137, Accuracy: 99.88%

Test set: Average loss: 0.5223, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.511091
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.509905
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.513427
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.507376
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.517740
Train Epoch: 73 - Loss: 0.5136, Accuracy: 99.89%

Test set: Average loss: 0.5223, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.511158
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.513936
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.519503
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.507685
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.509721
Train Epoch: 74 - Loss: 0.5138, Accuracy: 99.89%

Test set: Average loss: 0.5219, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.513317
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.514992
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.513948
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.521794
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.513083
Train Epoch: 75 - Loss: 0.5138, Accuracy: 99.88%

Test set: Average loss: 0.5220, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.512323
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.513170
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.510984
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.517145
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.516663
Train Epoch: 76 - Loss: 0.5151, Accuracy: 99.85%

Test set: Average loss: 0.5231, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.510797
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.524778
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.514738
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.513086
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.514682
Train Epoch: 77 - Loss: 0.5150, Accuracy: 99.84%

Test set: Average loss: 0.5219, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.509329
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.508013
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.516365
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.509542
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.508307
Train Epoch: 78 - Loss: 0.5144, Accuracy: 99.86%

Test set: Average loss: 0.5228, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.514426
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.511147
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.511158
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.512473
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.527802
Train Epoch: 79 - Loss: 0.5145, Accuracy: 99.86%

Test set: Average loss: 0.5223, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.522479
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.513170
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.512674
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.511216
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.523805
Train Epoch: 80 - Loss: 0.5141, Accuracy: 99.86%

Test set: Average loss: 0.5226, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.508089
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.509895
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.508234
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.509813
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.508819
Train Epoch: 81 - Loss: 0.5139, Accuracy: 99.86%

Test set: Average loss: 0.5223, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.519683
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.510328
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.509884
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.513128
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.513603
Train Epoch: 82 - Loss: 0.5137, Accuracy: 99.87%

Test set: Average loss: 0.5227, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.516345
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.511450
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.521172
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.506495
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.508789
Train Epoch: 83 - Loss: 0.5136, Accuracy: 99.88%

Test set: Average loss: 0.5223, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.510294
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.509599
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.515805
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.512709
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.509908
Train Epoch: 84 - Loss: 0.5131, Accuracy: 99.90%

Test set: Average loss: 0.5217, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.512791
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.509655
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.516140
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.515149
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.514838
Train Epoch: 85 - Loss: 0.5132, Accuracy: 99.88%

Test set: Average loss: 0.5212, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.512181
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.518958
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.512073
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.522415
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.514828
Train Epoch: 86 - Loss: 0.5128, Accuracy: 99.91%

Test set: Average loss: 0.5213, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.508020
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.514982
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.508119
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.521524
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.511283
Train Epoch: 87 - Loss: 0.5126, Accuracy: 99.89%

Test set: Average loss: 0.5217, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.506614
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507649
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.512100
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.518318
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.509452
Train Epoch: 88 - Loss: 0.5125, Accuracy: 99.91%

Test set: Average loss: 0.5212, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.509270
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.523025
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.507476
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.506405
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.512182
Train Epoch: 89 - Loss: 0.5122, Accuracy: 99.89%

Test set: Average loss: 0.5217, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.512860
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.507753
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.513977
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.517785
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.522120
Train Epoch: 90 - Loss: 0.5122, Accuracy: 99.91%

Test set: Average loss: 0.5211, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.511180
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.508447
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.513064
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.511027
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.513440
Train Epoch: 91 - Loss: 0.5117, Accuracy: 99.91%

Test set: Average loss: 0.5207, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.512026
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.513491
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.512755
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.505903
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.510562
Train Epoch: 92 - Loss: 0.5117, Accuracy: 99.92%

Test set: Average loss: 0.5207, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.511383
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.516066
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.527135
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.507903
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.509580
Train Epoch: 93 - Loss: 0.5116, Accuracy: 99.92%

Test set: Average loss: 0.5206, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.507704
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.515518
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.507963
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.515105
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.507668
Train Epoch: 94 - Loss: 0.5114, Accuracy: 99.95%

Test set: Average loss: 0.5210, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.516842
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.514755
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.512284
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.507639
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.513651
Train Epoch: 95 - Loss: 0.5113, Accuracy: 99.93%

Test set: Average loss: 0.5207, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.510098
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.509087
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.508790
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.509891
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.509864
Train Epoch: 96 - Loss: 0.5113, Accuracy: 99.94%

Test set: Average loss: 0.5208, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.514270
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.515678
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.508681
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.506078
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.514252
Train Epoch: 97 - Loss: 0.5113, Accuracy: 99.94%

Test set: Average loss: 0.5207, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.511486
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.506506
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508985
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508882
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.510326
Train Epoch: 98 - Loss: 0.5112, Accuracy: 99.93%

Test set: Average loss: 0.5205, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.510834
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.510760
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.513247
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.508188
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.511235
Train Epoch: 99 - Loss: 0.5112, Accuracy: 99.94%

Test set: Average loss: 0.5206, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.507920
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.515166
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.509883
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.509645
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.506306
Train Epoch: 100 - Loss: 0.5111, Accuracy: 99.95%

Test set: Average loss: 0.5205, Accuracy: 9945/10000 (99.45%)


Training completed. Best Accuracy: 99.46%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.324845
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.679129
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.624581
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.628689
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.602256
Train Epoch: 1 - Loss: 0.7115, Accuracy: 94.38%

Test set: Average loss: 0.6009, Accuracy: 9765/10000 (97.65%)

New best model saved with accuracy: 97.65%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.575606
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.593976
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.577170
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.588709
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.556275
Train Epoch: 2 - Loss: 0.5866, Accuracy: 98.12%

Test set: Average loss: 0.5692, Accuracy: 9847/10000 (98.47%)

New best model saved with accuracy: 98.47%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.547635
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.584736
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.563495
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.543688
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.553750
Train Epoch: 3 - Loss: 0.5685, Accuracy: 98.60%

Test set: Average loss: 0.5571, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.552622
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.541914
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.555987
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.539298
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.588689
Train Epoch: 4 - Loss: 0.5584, Accuracy: 98.85%

Test set: Average loss: 0.5505, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.566973
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.533338
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.549970
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.554999
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.535159
Train Epoch: 5 - Loss: 0.5512, Accuracy: 99.00%

Test set: Average loss: 0.5464, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.556879
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.580751
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.529877
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.562637
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.553538
Train Epoch: 6 - Loss: 0.5470, Accuracy: 99.11%

Test set: Average loss: 0.5429, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.533126
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.530979
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.575351
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.585100
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.552233
Train Epoch: 7 - Loss: 0.5431, Accuracy: 99.22%

Test set: Average loss: 0.5414, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.547077
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.543618
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.534708
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.541408
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.540828
Train Epoch: 8 - Loss: 0.5400, Accuracy: 99.31%

Test set: Average loss: 0.5390, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.532035
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.554455
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.526303
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.543197
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.569472
Train Epoch: 9 - Loss: 0.5382, Accuracy: 99.35%

Test set: Average loss: 0.5377, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.538351
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.533366
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.562683
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.526610
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.560993
Train Epoch: 10 - Loss: 0.5356, Accuracy: 99.39%

Test set: Average loss: 0.5364, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.527208
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.544469
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.524680
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.541012
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.530109
Train Epoch: 11 - Loss: 0.5347, Accuracy: 99.39%

Test set: Average loss: 0.5376, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.540640
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.525483
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.522898
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.522606
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.529647
Train Epoch: 12 - Loss: 0.5332, Accuracy: 99.45%

Test set: Average loss: 0.5344, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.524400
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.527431
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.531962
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.530753
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.534915
Train Epoch: 13 - Loss: 0.5319, Accuracy: 99.50%

Test set: Average loss: 0.5330, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.548544
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.520531
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.525311
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.519898
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.522304
Train Epoch: 14 - Loss: 0.5312, Accuracy: 99.52%

Test set: Average loss: 0.5330, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.536365
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.541093
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.517411
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.549957
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.536036
Train Epoch: 15 - Loss: 0.5301, Accuracy: 99.55%

Test set: Average loss: 0.5319, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.528484
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.523442
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.523917
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.524257
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.528486
Train Epoch: 16 - Loss: 0.5295, Accuracy: 99.54%

Test set: Average loss: 0.5321, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.526634
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.532432
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.516478
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.532750
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.530093
Train Epoch: 17 - Loss: 0.5291, Accuracy: 99.56%

Test set: Average loss: 0.5312, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.520518
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.520680
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.515924
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.526198
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.517159
Train Epoch: 18 - Loss: 0.5284, Accuracy: 99.56%

Test set: Average loss: 0.5317, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.530678
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.521900
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.544679
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.522855
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.550592
Train Epoch: 19 - Loss: 0.5276, Accuracy: 99.60%

Test set: Average loss: 0.5312, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.527308
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.531110
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.525454
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.523939
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.520247
Train Epoch: 20 - Loss: 0.5275, Accuracy: 99.60%

Test set: Average loss: 0.5308, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.527012
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.539967
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.520227
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.524758
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.527238
Train Epoch: 21 - Loss: 0.5273, Accuracy: 99.62%

Test set: Average loss: 0.5310, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.534197
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.522411
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.531352
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.534326
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.531584
Train Epoch: 22 - Loss: 0.5271, Accuracy: 99.63%

Test set: Average loss: 0.5307, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.518716
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.529214
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.523663
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.545943
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.526088
Train Epoch: 23 - Loss: 0.5268, Accuracy: 99.62%

Test set: Average loss: 0.5306, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.526290
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.531114
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.529870
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.521306
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.526612
Train Epoch: 24 - Loss: 0.5269, Accuracy: 99.64%

Test set: Average loss: 0.5304, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.523985
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.538739
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.520638
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.528271
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.524334
Train Epoch: 25 - Loss: 0.5270, Accuracy: 99.62%

Test set: Average loss: 0.5307, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.529278
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.524382
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.530175
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.529323
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.535154
Train Epoch: 26 - Loss: 0.5296, Accuracy: 99.51%

Test set: Average loss: 0.5390, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.523646
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.515185
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.524279
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.520623
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.519491
Train Epoch: 27 - Loss: 0.5280, Accuracy: 99.56%

Test set: Average loss: 0.5312, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.528540
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.531820
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.529023
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.524740
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.526919
Train Epoch: 28 - Loss: 0.5272, Accuracy: 99.60%

Test set: Average loss: 0.5298, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.530287
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.520782
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.515068
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.515821
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.518181
Train Epoch: 29 - Loss: 0.5261, Accuracy: 99.58%

Test set: Average loss: 0.5298, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.528932
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.551667
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.525008
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.528661
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.534211
Train Epoch: 30 - Loss: 0.5251, Accuracy: 99.60%

Test set: Average loss: 0.5282, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.515642
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.529158
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.517059
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.525490
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.514626
Train Epoch: 31 - Loss: 0.5243, Accuracy: 99.64%

Test set: Average loss: 0.5278, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.519442
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.510378
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.517461
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.526082
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.522340
Train Epoch: 32 - Loss: 0.5237, Accuracy: 99.67%

Test set: Average loss: 0.5278, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.516246
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.515400
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.521021
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.527652
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.523275
Train Epoch: 33 - Loss: 0.5229, Accuracy: 99.69%

Test set: Average loss: 0.5271, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.513217
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.523529
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.512115
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.517847
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.528090
Train Epoch: 34 - Loss: 0.5223, Accuracy: 99.67%

Test set: Average loss: 0.5271, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.519718
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.518001
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.517436
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.522881
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.514552
Train Epoch: 35 - Loss: 0.5217, Accuracy: 99.70%

Test set: Average loss: 0.5262, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.516411
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.522523
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.534539
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.512979
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.528996
Train Epoch: 36 - Loss: 0.5211, Accuracy: 99.70%

Test set: Average loss: 0.5256, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.516118
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.525628
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.515143
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.515263
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.512346
Train Epoch: 37 - Loss: 0.5207, Accuracy: 99.71%

Test set: Average loss: 0.5254, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.518267
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.513342
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.516161
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.515492
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.524776
Train Epoch: 38 - Loss: 0.5200, Accuracy: 99.74%

Test set: Average loss: 0.5253, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.512063
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.517792
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.512303
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.524087
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.538727
Train Epoch: 39 - Loss: 0.5195, Accuracy: 99.76%

Test set: Average loss: 0.5255, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.519540
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.516207
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.514785
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.516298
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.520537
Train Epoch: 40 - Loss: 0.5193, Accuracy: 99.77%

Test set: Average loss: 0.5247, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.515016
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.517637
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.518415
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.516439
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.519688
Train Epoch: 41 - Loss: 0.5189, Accuracy: 99.78%

Test set: Average loss: 0.5249, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.518179
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.524161
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.529228
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.522327
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.513384
Train Epoch: 42 - Loss: 0.5188, Accuracy: 99.78%

Test set: Average loss: 0.5248, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.511686
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.533725
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.517283
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.523816
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.514331
Train Epoch: 43 - Loss: 0.5184, Accuracy: 99.78%

Test set: Average loss: 0.5242, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.510685
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.514041
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.514775
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.519771
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.518428
Train Epoch: 44 - Loss: 0.5180, Accuracy: 99.82%

Test set: Average loss: 0.5241, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.511059
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.521785
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.526830
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.522915
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.547496
Train Epoch: 45 - Loss: 0.5182, Accuracy: 99.80%

Test set: Average loss: 0.5242, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.513018
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.517520
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.518586
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.518541
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.522179
Train Epoch: 46 - Loss: 0.5181, Accuracy: 99.78%

Test set: Average loss: 0.5241, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.514447
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.521564
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.515470
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.530002
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.516447
Train Epoch: 47 - Loss: 0.5180, Accuracy: 99.81%

Test set: Average loss: 0.5240, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.513435
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.514310
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.516824
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.513969
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.510912
Train Epoch: 48 - Loss: 0.5180, Accuracy: 99.78%

Test set: Average loss: 0.5242, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.514590
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.514043
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.517220
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.514436
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.518930
Train Epoch: 49 - Loss: 0.5178, Accuracy: 99.80%

Test set: Average loss: 0.5239, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.511480
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.518208
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.509786
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.513971
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.513763
Train Epoch: 50 - Loss: 0.5178, Accuracy: 99.81%

Test set: Average loss: 0.5240, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.517803
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.511678
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.523435
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.550899
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.518664
Train Epoch: 51 - Loss: 0.5197, Accuracy: 99.75%

Test set: Average loss: 0.5265, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.515085
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.523532
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.508830
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.516239
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.539552
Train Epoch: 52 - Loss: 0.5193, Accuracy: 99.75%

Test set: Average loss: 0.5259, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.511616
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.516539
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.522196
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.528622
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.513012
Train Epoch: 53 - Loss: 0.5189, Accuracy: 99.76%

Test set: Average loss: 0.5247, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.518633
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.528389
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.512942
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.513665
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.516667
Train Epoch: 54 - Loss: 0.5184, Accuracy: 99.77%

Test set: Average loss: 0.5260, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.519465
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.513246
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.537179
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.526610
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.516312
Train Epoch: 55 - Loss: 0.5178, Accuracy: 99.80%

Test set: Average loss: 0.5243, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.524215
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.529762
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.524979
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.511023
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.511411
Train Epoch: 56 - Loss: 0.5174, Accuracy: 99.80%

Test set: Average loss: 0.5234, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.518431
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.520555
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.521180
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.509532
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.514144
Train Epoch: 57 - Loss: 0.5171, Accuracy: 99.79%

Test set: Average loss: 0.5235, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.513315
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.515324
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.512896
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.514484
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.509103
Train Epoch: 58 - Loss: 0.5167, Accuracy: 99.82%

Test set: Average loss: 0.5233, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.511060
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.513717
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.517405
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.518202
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.511429
Train Epoch: 59 - Loss: 0.5166, Accuracy: 99.80%

Test set: Average loss: 0.5228, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.510870
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.513221
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.509824
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.521195
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.510381
Train Epoch: 60 - Loss: 0.5160, Accuracy: 99.81%

Test set: Average loss: 0.5225, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.512196
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.516041
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.525776
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.520581
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.515398
Train Epoch: 61 - Loss: 0.5158, Accuracy: 99.83%

Test set: Average loss: 0.5233, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.532238
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.519340
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.531695
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.515661
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.509971
Train Epoch: 62 - Loss: 0.5154, Accuracy: 99.83%

Test set: Average loss: 0.5220, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.516520
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.506198
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.510829
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.512438
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.511760
Train Epoch: 63 - Loss: 0.5151, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.515296
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.511069
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.510301
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.516664
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.516728
Train Epoch: 64 - Loss: 0.5149, Accuracy: 99.84%

Test set: Average loss: 0.5225, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.512259
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.510502
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.511425
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.516948
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.515713
Train Epoch: 65 - Loss: 0.5147, Accuracy: 99.87%

Test set: Average loss: 0.5218, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.511399
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.516423
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.518645
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.513163
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.511344
Train Epoch: 66 - Loss: 0.5144, Accuracy: 99.85%

Test set: Average loss: 0.5221, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.516155
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.516310
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.516056
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.516124
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513331
Train Epoch: 67 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5220, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.510085
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.515010
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.515408
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.510498
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.514536
Train Epoch: 68 - Loss: 0.5144, Accuracy: 99.85%

Test set: Average loss: 0.5222, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.515731
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.515913
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.508193
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.509381
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.514462
Train Epoch: 69 - Loss: 0.5141, Accuracy: 99.87%

Test set: Average loss: 0.5218, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.510669
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.520887
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.511275
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.516014
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.511392
Train Epoch: 70 - Loss: 0.5139, Accuracy: 99.87%

Test set: Average loss: 0.5216, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.507453
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.509888
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.509570
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.509120
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.521966
Train Epoch: 71 - Loss: 0.5141, Accuracy: 99.87%

Test set: Average loss: 0.5217, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.515000
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509209
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.509949
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.508034
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.510285
Train Epoch: 72 - Loss: 0.5139, Accuracy: 99.88%

Test set: Average loss: 0.5217, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.521756
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.510683
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.518304
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.510468
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.510540
Train Epoch: 73 - Loss: 0.5138, Accuracy: 99.88%

Test set: Average loss: 0.5218, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.513544
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.516068
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.518349
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.514380
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.519333
Train Epoch: 74 - Loss: 0.5138, Accuracy: 99.87%

Test set: Average loss: 0.5217, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.511220
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.517106
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.515888
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.512216
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.514733
Train Epoch: 75 - Loss: 0.5135, Accuracy: 99.89%

Test set: Average loss: 0.5215, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.507921
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.512655
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.510503
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.519834
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.513842
Train Epoch: 76 - Loss: 0.5150, Accuracy: 99.84%

Test set: Average loss: 0.5226, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.510631
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.508815
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.510995
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.515786
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.512838
Train Epoch: 77 - Loss: 0.5152, Accuracy: 99.82%

Test set: Average loss: 0.5219, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.510773
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.509867
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.512849
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.510171
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.513814
Train Epoch: 78 - Loss: 0.5146, Accuracy: 99.84%

Test set: Average loss: 0.5229, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.511152
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.513379
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.525381
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.517434
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.514599
Train Epoch: 79 - Loss: 0.5143, Accuracy: 99.86%

Test set: Average loss: 0.5216, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.506522
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.513900
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.509365
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.513380
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.531509
Train Epoch: 80 - Loss: 0.5141, Accuracy: 99.85%

Test set: Average loss: 0.5216, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.515139
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.515713
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.510183
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.511645
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.528700
Train Epoch: 81 - Loss: 0.5138, Accuracy: 99.85%

Test set: Average loss: 0.5215, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.512550
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.520414
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.524308
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.517824
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.513870
Train Epoch: 82 - Loss: 0.5134, Accuracy: 99.87%

Test set: Average loss: 0.5210, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.517508
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512104
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.514113
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.519384
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.513300
Train Epoch: 83 - Loss: 0.5134, Accuracy: 99.86%

Test set: Average loss: 0.5216, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.510515
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.508854
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.513216
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.520047
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.514607
Train Epoch: 84 - Loss: 0.5132, Accuracy: 99.88%

Test set: Average loss: 0.5214, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.508188
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.516668
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.508154
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.509863
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.515513
Train Epoch: 85 - Loss: 0.5128, Accuracy: 99.88%

Test set: Average loss: 0.5210, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.511433
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.511505
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.508649
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.510825
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.510777
Train Epoch: 86 - Loss: 0.5123, Accuracy: 99.91%

Test set: Average loss: 0.5218, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.509986
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.514583
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.510520
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.509779
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.516583
Train Epoch: 87 - Loss: 0.5124, Accuracy: 99.89%

Test set: Average loss: 0.5205, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.511789
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.508946
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509100
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.508278
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.509678
Train Epoch: 88 - Loss: 0.5121, Accuracy: 99.91%

Test set: Average loss: 0.5209, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.517699
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.510828
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.507940
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.518061
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.517143
Train Epoch: 89 - Loss: 0.5121, Accuracy: 99.91%

Test set: Average loss: 0.5206, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.514383
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.507582
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.507692
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.511473
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.511914
Train Epoch: 90 - Loss: 0.5118, Accuracy: 99.92%

Test set: Average loss: 0.5209, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.510097
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.510473
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.513830
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.511814
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.515769
Train Epoch: 91 - Loss: 0.5118, Accuracy: 99.90%

Test set: Average loss: 0.5203, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.508847
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.518224
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.512741
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.517420
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.515679
Train Epoch: 92 - Loss: 0.5114, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.520781
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.517806
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.510650
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.512780
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.510310
Train Epoch: 93 - Loss: 0.5115, Accuracy: 99.91%

Test set: Average loss: 0.5203, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.508211
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.508014
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.507243
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.510942
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.508253
Train Epoch: 94 - Loss: 0.5113, Accuracy: 99.92%

Test set: Average loss: 0.5201, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.511694
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.508116
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506974
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.516598
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.508127
Train Epoch: 95 - Loss: 0.5114, Accuracy: 99.92%

Test set: Average loss: 0.5203, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.506879
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.509415
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.510230
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.507970
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.508123
Train Epoch: 96 - Loss: 0.5113, Accuracy: 99.92%

Test set: Average loss: 0.5203, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.511670
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.516283
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.513584
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.508634
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.508257
Train Epoch: 97 - Loss: 0.5111, Accuracy: 99.93%

Test set: Average loss: 0.5203, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.507437
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.511864
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.506129
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.507660
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.507521
Train Epoch: 98 - Loss: 0.5111, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.535146
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.507572
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.514200
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.506906
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.509520
Train Epoch: 99 - Loss: 0.5111, Accuracy: 99.92%

Test set: Average loss: 0.5201, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.513430
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.511276
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.511218
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.511286
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.511484
Train Epoch: 100 - Loss: 0.5111, Accuracy: 99.93%

Test set: Average loss: 0.5202, Accuracy: 9948/10000 (99.48%)


Training completed. Best Accuracy: 99.50%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.580071
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.707433
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.649110
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.586200
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.630266
Train Epoch: 1 - Loss: 0.7142, Accuracy: 94.33%

Test set: Average loss: 0.5947, Accuracy: 9780/10000 (97.80%)

New best model saved with accuracy: 97.80%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.578056
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.656598
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.581133
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.617624
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.592525
Train Epoch: 2 - Loss: 0.5884, Accuracy: 98.04%

Test set: Average loss: 0.5672, Accuracy: 9852/10000 (98.52%)

New best model saved with accuracy: 98.52%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.570174
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.575698
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.611463
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.546373
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.573205
Train Epoch: 3 - Loss: 0.5689, Accuracy: 98.56%

Test set: Average loss: 0.5665, Accuracy: 9853/10000 (98.53%)

New best model saved with accuracy: 98.53%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.556080
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.546234
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.561260
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.572579
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.544333
Train Epoch: 4 - Loss: 0.5592, Accuracy: 98.81%

Test set: Average loss: 0.5544, Accuracy: 9879/10000 (98.79%)

New best model saved with accuracy: 98.79%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.546566
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.551790
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.554279
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.532330
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.594793
Train Epoch: 5 - Loss: 0.5521, Accuracy: 99.00%

Test set: Average loss: 0.5452, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.540163
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.538044
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.545254
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.542706
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.532732
Train Epoch: 6 - Loss: 0.5481, Accuracy: 99.06%

Test set: Average loss: 0.5451, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.539730
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.535129
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.540361
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.521005
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.532934
Train Epoch: 7 - Loss: 0.5438, Accuracy: 99.17%

Test set: Average loss: 0.5387, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.538951
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.542483
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.544741
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.539786
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.547968
Train Epoch: 8 - Loss: 0.5405, Accuracy: 99.23%

Test set: Average loss: 0.5407, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.535311
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.533961
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.550350
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.522215
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.528779
Train Epoch: 9 - Loss: 0.5384, Accuracy: 99.32%

Test set: Average loss: 0.5369, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.583723
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.540607
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.534415
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.539009
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.518644
Train Epoch: 10 - Loss: 0.5362, Accuracy: 99.37%

Test set: Average loss: 0.5385, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.528328
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.573506
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.541882
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.547652
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.543153
Train Epoch: 11 - Loss: 0.5348, Accuracy: 99.43%

Test set: Average loss: 0.5359, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.543744
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.527968
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.526982
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.522465
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.554384
Train Epoch: 12 - Loss: 0.5332, Accuracy: 99.48%

Test set: Average loss: 0.5336, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.527725
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.517419
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.525186
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.526727
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.530304
Train Epoch: 13 - Loss: 0.5320, Accuracy: 99.49%

Test set: Average loss: 0.5335, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.551663
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.525235
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.524083
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.547368
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.543831
Train Epoch: 14 - Loss: 0.5310, Accuracy: 99.51%

Test set: Average loss: 0.5321, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.520518
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.527164
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.531986
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.519825
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.518370
Train Epoch: 15 - Loss: 0.5301, Accuracy: 99.53%

Test set: Average loss: 0.5312, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.521314
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.516713
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.522732
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.516737
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.533510
Train Epoch: 16 - Loss: 0.5295, Accuracy: 99.56%

Test set: Average loss: 0.5306, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.535847
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.520168
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.533106
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.522487
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.519681
Train Epoch: 17 - Loss: 0.5288, Accuracy: 99.55%

Test set: Average loss: 0.5310, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.538392
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.585060
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.518342
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.519446
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.524825
Train Epoch: 18 - Loss: 0.5283, Accuracy: 99.58%

Test set: Average loss: 0.5303, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.535673
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.521361
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.519970
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.521147
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.527620
Train Epoch: 19 - Loss: 0.5277, Accuracy: 99.58%

Test set: Average loss: 0.5301, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.531052
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.519687
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.531244
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.537517
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.536163
Train Epoch: 20 - Loss: 0.5271, Accuracy: 99.62%

Test set: Average loss: 0.5300, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.526250
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.523522
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.531866
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.521958
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.525756
Train Epoch: 21 - Loss: 0.5273, Accuracy: 99.60%

Test set: Average loss: 0.5298, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.513877
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.521251
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.520331
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.518355
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.540552
Train Epoch: 22 - Loss: 0.5269, Accuracy: 99.63%

Test set: Average loss: 0.5296, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.531500
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.519972
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.531299
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.517364
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.528010
Train Epoch: 23 - Loss: 0.5269, Accuracy: 99.61%

Test set: Average loss: 0.5296, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.521070
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.522305
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.526089
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.529514
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.531276
Train Epoch: 24 - Loss: 0.5261, Accuracy: 99.65%

Test set: Average loss: 0.5297, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.542706
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.525820
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.526868
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.539288
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.523562
Train Epoch: 25 - Loss: 0.5266, Accuracy: 99.64%

Test set: Average loss: 0.5297, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.532785
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.526978
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.520884
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.525600
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.525025
Train Epoch: 26 - Loss: 0.5295, Accuracy: 99.51%

Test set: Average loss: 0.5322, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.519950
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.523577
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.520698
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.546708
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.545358
Train Epoch: 27 - Loss: 0.5285, Accuracy: 99.55%

Test set: Average loss: 0.5301, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.534715
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.523376
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.522902
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.536797
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.521870
Train Epoch: 28 - Loss: 0.5271, Accuracy: 99.58%

Test set: Average loss: 0.5303, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.516821
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.519111
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.533341
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.517503
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.522577
Train Epoch: 29 - Loss: 0.5262, Accuracy: 99.60%

Test set: Average loss: 0.5300, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.513385
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.523960
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.539274
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.526079
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.517616
Train Epoch: 30 - Loss: 0.5249, Accuracy: 99.63%

Test set: Average loss: 0.5297, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.515497
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.520264
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.521417
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.531509
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.533076
Train Epoch: 31 - Loss: 0.5245, Accuracy: 99.63%

Test set: Average loss: 0.5275, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.525195
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.524277
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.516062
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.521174
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.523139
Train Epoch: 32 - Loss: 0.5237, Accuracy: 99.65%

Test set: Average loss: 0.5283, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.518452
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.519625
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.527912
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.529287
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.513911
Train Epoch: 33 - Loss: 0.5226, Accuracy: 99.68%

Test set: Average loss: 0.5273, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.537804
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.511708
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.519390
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.522329
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.524588
Train Epoch: 34 - Loss: 0.5228, Accuracy: 99.65%

Test set: Average loss: 0.5263, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.518495
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.513393
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.521126
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.515552
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.518793
Train Epoch: 35 - Loss: 0.5219, Accuracy: 99.70%

Test set: Average loss: 0.5366, Accuracy: 9901/10000 (99.01%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.529540
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.518925
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.542062
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.523534
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.525722
Train Epoch: 36 - Loss: 0.5211, Accuracy: 99.73%

Test set: Average loss: 0.5261, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.513614
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.516760
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.528490
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.516418
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.510923
Train Epoch: 37 - Loss: 0.5207, Accuracy: 99.73%

Test set: Average loss: 0.5248, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.515803
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.516307
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.514348
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.514367
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.526016
Train Epoch: 38 - Loss: 0.5203, Accuracy: 99.77%

Test set: Average loss: 0.5253, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.528260
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.525234
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.516791
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.528724
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.514407
Train Epoch: 39 - Loss: 0.5200, Accuracy: 99.75%

Test set: Average loss: 0.5247, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.511303
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.535147
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.526756
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.510488
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.524563
Train Epoch: 40 - Loss: 0.5195, Accuracy: 99.76%

Test set: Average loss: 0.5249, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.518126
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.517683
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.515467
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.511853
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.525754
Train Epoch: 41 - Loss: 0.5191, Accuracy: 99.78%

Test set: Average loss: 0.5246, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.534734
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.513890
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.526922
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.512292
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.514472
Train Epoch: 42 - Loss: 0.5190, Accuracy: 99.78%

Test set: Average loss: 0.5244, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.514671
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.517740
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.523933
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.516522
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.517333
Train Epoch: 43 - Loss: 0.5183, Accuracy: 99.81%

Test set: Average loss: 0.5245, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.516023
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.513318
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.511956
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.526198
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.518337
Train Epoch: 44 - Loss: 0.5185, Accuracy: 99.78%

Test set: Average loss: 0.5242, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.514838
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.512960
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.516181
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.522175
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.519825
Train Epoch: 45 - Loss: 0.5182, Accuracy: 99.81%

Test set: Average loss: 0.5243, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.515027
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.516553
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.512273
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.530096
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.512668
Train Epoch: 46 - Loss: 0.5179, Accuracy: 99.82%

Test set: Average loss: 0.5239, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.511721
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.511764
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.516055
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.515100
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.514496
Train Epoch: 47 - Loss: 0.5178, Accuracy: 99.82%

Test set: Average loss: 0.5240, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.515057
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.524337
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.514029
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.519933
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.520393
Train Epoch: 48 - Loss: 0.5180, Accuracy: 99.80%

Test set: Average loss: 0.5240, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.515880
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.518361
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.509371
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.519640
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.512288
Train Epoch: 49 - Loss: 0.5177, Accuracy: 99.81%

Test set: Average loss: 0.5241, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.513741
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.515957
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.512706
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.509779
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.518900
Train Epoch: 50 - Loss: 0.5177, Accuracy: 99.80%

Test set: Average loss: 0.5240, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.523920
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.521407
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.517265
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.524620
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.530466
Train Epoch: 51 - Loss: 0.5200, Accuracy: 99.70%

Test set: Average loss: 0.5253, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.515818
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.518687
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.511091
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.514926
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.521814
Train Epoch: 52 - Loss: 0.5195, Accuracy: 99.75%

Test set: Average loss: 0.5266, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.514521
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.526953
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.522506
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.513662
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.522164
Train Epoch: 53 - Loss: 0.5190, Accuracy: 99.75%

Test set: Average loss: 0.5247, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.514490
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.521451
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.518250
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.526702
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.513260
Train Epoch: 54 - Loss: 0.5183, Accuracy: 99.80%

Test set: Average loss: 0.5257, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.511190
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.520999
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.516052
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.512554
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.516612
Train Epoch: 55 - Loss: 0.5180, Accuracy: 99.78%

Test set: Average loss: 0.5251, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.514836
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.520168
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.524961
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.515957
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.516364
Train Epoch: 56 - Loss: 0.5182, Accuracy: 99.75%

Test set: Average loss: 0.5244, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.517341
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.519027
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.522552
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.514198
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.520986
Train Epoch: 57 - Loss: 0.5171, Accuracy: 99.81%

Test set: Average loss: 0.5232, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.518270
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.518857
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.511886
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.519095
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.520838
Train Epoch: 58 - Loss: 0.5167, Accuracy: 99.82%

Test set: Average loss: 0.5245, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.517239
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.512642
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.515230
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.515653
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.511668
Train Epoch: 59 - Loss: 0.5167, Accuracy: 99.80%

Test set: Average loss: 0.5233, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.522119
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.510925
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.514215
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.520200
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.515593
Train Epoch: 60 - Loss: 0.5162, Accuracy: 99.85%

Test set: Average loss: 0.5232, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.523935
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.510270
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.521874
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.513030
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.510551
Train Epoch: 61 - Loss: 0.5159, Accuracy: 99.83%

Test set: Average loss: 0.5226, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.518252
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.512756
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.511233
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.515085
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.513828
Train Epoch: 62 - Loss: 0.5154, Accuracy: 99.86%

Test set: Average loss: 0.5223, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.529877
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.510849
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.515526
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.509780
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.512988
Train Epoch: 63 - Loss: 0.5152, Accuracy: 99.85%

Test set: Average loss: 0.5222, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.512701
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.519917
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.517431
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.508605
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.526447
Train Epoch: 64 - Loss: 0.5151, Accuracy: 99.84%

Test set: Average loss: 0.5219, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.512595
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.533844
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.510033
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.511978
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.514263
Train Epoch: 65 - Loss: 0.5150, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.534616
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.514181
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.512376
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.509155
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.509570
Train Epoch: 66 - Loss: 0.5146, Accuracy: 99.87%

Test set: Average loss: 0.5217, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.517248
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.518372
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.509805
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.511227
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.511207
Train Epoch: 67 - Loss: 0.5144, Accuracy: 99.86%

Test set: Average loss: 0.5217, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.513325
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.509536
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.530756
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.513496
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.510353
Train Epoch: 68 - Loss: 0.5141, Accuracy: 99.88%

Test set: Average loss: 0.5217, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.511195
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.514150
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.514147
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.509915
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.510749
Train Epoch: 69 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5216, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.507577
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.512480
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.516741
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.517519
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.526594
Train Epoch: 70 - Loss: 0.5140, Accuracy: 99.87%

Test set: Average loss: 0.5217, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.525934
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.511360
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.517186
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.511814
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.516050
Train Epoch: 71 - Loss: 0.5139, Accuracy: 99.89%

Test set: Average loss: 0.5215, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.520563
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.510492
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.511942
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.512514
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.507434
Train Epoch: 72 - Loss: 0.5137, Accuracy: 99.89%

Test set: Average loss: 0.5216, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.509466
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.513832
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.514773
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.515348
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.518151
Train Epoch: 73 - Loss: 0.5136, Accuracy: 99.90%

Test set: Average loss: 0.5215, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.517882
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.518520
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.510201
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.508612
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.517703
Train Epoch: 74 - Loss: 0.5137, Accuracy: 99.90%

Test set: Average loss: 0.5214, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.509106
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.508005
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.511883
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.510004
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.508586
Train Epoch: 75 - Loss: 0.5138, Accuracy: 99.91%

Test set: Average loss: 0.5214, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.515113
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.508925
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.514375
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.511415
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.544076
Train Epoch: 76 - Loss: 0.5155, Accuracy: 99.82%

Test set: Average loss: 0.5228, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.513850
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.510165
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.512536
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.511632
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.513401
Train Epoch: 77 - Loss: 0.5151, Accuracy: 99.84%

Test set: Average loss: 0.5235, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.520977
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.514528
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.526505
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.517921
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.525685
Train Epoch: 78 - Loss: 0.5149, Accuracy: 99.84%

Test set: Average loss: 0.5245, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.516840
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.517477
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.506332
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.508268
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.508549
Train Epoch: 79 - Loss: 0.5148, Accuracy: 99.83%

Test set: Average loss: 0.5216, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.509155
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.516755
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.507917
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.515062
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.526294
Train Epoch: 80 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5213, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.508512
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.512559
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.517196
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.508912
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.512429
Train Epoch: 81 - Loss: 0.5141, Accuracy: 99.88%

Test set: Average loss: 0.5226, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.508368
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.510412
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.515792
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.509956
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.514822
Train Epoch: 82 - Loss: 0.5134, Accuracy: 99.88%

Test set: Average loss: 0.5211, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.508886
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.515422
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.509279
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.511773
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.508776
Train Epoch: 83 - Loss: 0.5134, Accuracy: 99.88%

Test set: Average loss: 0.5208, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.519046
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.516577
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.513216
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.514419
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.507990
Train Epoch: 84 - Loss: 0.5133, Accuracy: 99.88%

Test set: Average loss: 0.5210, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.519979
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.513827
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.519078
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.508457
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.515669
Train Epoch: 85 - Loss: 0.5129, Accuracy: 99.90%

Test set: Average loss: 0.5208, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.512322
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.515495
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.512698
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.512799
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.507501
Train Epoch: 86 - Loss: 0.5126, Accuracy: 99.89%

Test set: Average loss: 0.5206, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.513916
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.506046
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.511319
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.508640
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.514509
Train Epoch: 87 - Loss: 0.5126, Accuracy: 99.90%

Test set: Average loss: 0.5210, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.512699
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.517237
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.510054
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.509799
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.519445
Train Epoch: 88 - Loss: 0.5123, Accuracy: 99.92%

Test set: Average loss: 0.5203, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.512666
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.509758
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.512306
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.508394
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.511924
Train Epoch: 89 - Loss: 0.5123, Accuracy: 99.91%

Test set: Average loss: 0.5207, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.508070
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513364
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.516458
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.510055
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.507288
Train Epoch: 90 - Loss: 0.5120, Accuracy: 99.90%

Test set: Average loss: 0.5201, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.511379
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.509475
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.509763
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.508990
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.506029
Train Epoch: 91 - Loss: 0.5120, Accuracy: 99.91%

Test set: Average loss: 0.5201, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.514148
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.508877
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.506345
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507214
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.520628
Train Epoch: 92 - Loss: 0.5118, Accuracy: 99.90%

Test set: Average loss: 0.5198, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.509806
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.510330
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.511598
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.510195
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.507058
Train Epoch: 93 - Loss: 0.5116, Accuracy: 99.93%

Test set: Average loss: 0.5200, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.512701
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.514592
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.508185
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.511998
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.513797
Train Epoch: 94 - Loss: 0.5116, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.516230
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.511479
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.507860
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.512035
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.509457
Train Epoch: 95 - Loss: 0.5115, Accuracy: 99.94%

Test set: Average loss: 0.5199, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.509954
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508118
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.517809
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.509766
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.537733
Train Epoch: 96 - Loss: 0.5114, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.515048
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508476
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.511146
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.506674
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.519561
Train Epoch: 97 - Loss: 0.5113, Accuracy: 99.93%

Test set: Average loss: 0.5199, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.521168
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.505634
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508509
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511931
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.506030
Train Epoch: 98 - Loss: 0.5112, Accuracy: 99.93%

Test set: Average loss: 0.5198, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.507752
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.517155
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.508037
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.521517
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.510442
Train Epoch: 99 - Loss: 0.5111, Accuracy: 99.94%

Test set: Average loss: 0.5199, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.509204
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.520481
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.508011
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.510845
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.510368
Train Epoch: 100 - Loss: 0.5112, Accuracy: 99.94%

Test set: Average loss: 0.5200, Accuracy: 9942/10000 (99.42%)


Training completed. Best Accuracy: 99.44%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.031692
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.706558
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.639603
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.590671
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.584512
Train Epoch: 1 - Loss: 0.7120, Accuracy: 94.44%

Test set: Average loss: 0.6069, Accuracy: 9760/10000 (97.60%)

New best model saved with accuracy: 97.60%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.664760
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.618307
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.580042
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.594968
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.563453
Train Epoch: 2 - Loss: 0.5885, Accuracy: 98.01%

Test set: Average loss: 0.5828, Accuracy: 9796/10000 (97.96%)

New best model saved with accuracy: 97.96%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.568328
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.556919
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.559499
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.557581
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.604610
Train Epoch: 3 - Loss: 0.5697, Accuracy: 98.55%

Test set: Average loss: 0.5613, Accuracy: 9851/10000 (98.51%)

New best model saved with accuracy: 98.51%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.601288
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.570712
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.628587
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.556122
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.539801
Train Epoch: 4 - Loss: 0.5597, Accuracy: 98.75%

Test set: Average loss: 0.5536, Accuracy: 9876/10000 (98.76%)

New best model saved with accuracy: 98.76%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.553552
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.628092
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.535137
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.544461
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.582840
Train Epoch: 5 - Loss: 0.5531, Accuracy: 98.93%

Test set: Average loss: 0.5473, Accuracy: 9892/10000 (98.92%)

New best model saved with accuracy: 98.92%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.546310
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.553568
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.543499
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.554263
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.554805
Train Epoch: 6 - Loss: 0.5481, Accuracy: 99.04%

Test set: Average loss: 0.5429, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.548563
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.530626
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.534561
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.540298
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.542129
Train Epoch: 7 - Loss: 0.5438, Accuracy: 99.19%

Test set: Average loss: 0.5449, Accuracy: 9891/10000 (98.91%)

Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.562723
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.526867
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.543524
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.541308
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.529236
Train Epoch: 8 - Loss: 0.5412, Accuracy: 99.22%

Test set: Average loss: 0.5425, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.533362
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.532442
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.523844
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.550145
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.546224
Train Epoch: 9 - Loss: 0.5390, Accuracy: 99.28%

Test set: Average loss: 0.5399, Accuracy: 9902/10000 (99.02%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.543370
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.536597
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.527808
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.537354
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.526100
Train Epoch: 10 - Loss: 0.5366, Accuracy: 99.31%

Test set: Average loss: 0.5382, Accuracy: 9904/10000 (99.04%)

New best model saved with accuracy: 99.04%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.556859
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.518870
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.568557
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.534262
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.535304
Train Epoch: 11 - Loss: 0.5353, Accuracy: 99.33%

Test set: Average loss: 0.5351, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.550870
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.523393
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.518141
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.543329
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.549173
Train Epoch: 12 - Loss: 0.5335, Accuracy: 99.41%

Test set: Average loss: 0.5355, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.545269
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.543816
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.520786
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.521072
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.570918
Train Epoch: 13 - Loss: 0.5326, Accuracy: 99.42%

Test set: Average loss: 0.5328, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.521389
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.526933
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.546974
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.525878
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.517443
Train Epoch: 14 - Loss: 0.5316, Accuracy: 99.45%

Test set: Average loss: 0.5329, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.532058
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.530809
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.543208
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.541415
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.545226
Train Epoch: 15 - Loss: 0.5306, Accuracy: 99.46%

Test set: Average loss: 0.5330, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.522492
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.527342
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.515745
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.531636
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.539253
Train Epoch: 16 - Loss: 0.5302, Accuracy: 99.51%

Test set: Average loss: 0.5307, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.528744
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.520119
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.526647
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.532624
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.516020
Train Epoch: 17 - Loss: 0.5293, Accuracy: 99.52%

Test set: Average loss: 0.5307, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.550010
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.517596
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.523927
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.544737
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.550083
Train Epoch: 18 - Loss: 0.5286, Accuracy: 99.57%

Test set: Average loss: 0.5300, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.524707
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.523760
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.548174
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.516160
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.536553
Train Epoch: 19 - Loss: 0.5282, Accuracy: 99.56%

Test set: Average loss: 0.5296, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.534391
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.543621
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.524828
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.553241
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.541790
Train Epoch: 20 - Loss: 0.5280, Accuracy: 99.56%

Test set: Average loss: 0.5297, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.517542
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.516832
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.520100
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.520739
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.542756
Train Epoch: 21 - Loss: 0.5278, Accuracy: 99.56%

Test set: Average loss: 0.5298, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.532414
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.530102
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.522123
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.524540
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.527612
Train Epoch: 22 - Loss: 0.5276, Accuracy: 99.56%

Test set: Average loss: 0.5294, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.530965
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.520672
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.526744
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.537514
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.516259
Train Epoch: 23 - Loss: 0.5274, Accuracy: 99.57%

Test set: Average loss: 0.5298, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.533846
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.527297
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.526312
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.522353
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.517307
Train Epoch: 24 - Loss: 0.5270, Accuracy: 99.60%

Test set: Average loss: 0.5292, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.514601
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.531707
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.527353
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.522344
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.525740
Train Epoch: 25 - Loss: 0.5271, Accuracy: 99.59%

Test set: Average loss: 0.5292, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.520437
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.529363
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.548296
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.526585
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.537053
Train Epoch: 26 - Loss: 0.5300, Accuracy: 99.50%

Test set: Average loss: 0.5318, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.526160
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.526654
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.531870
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.531493
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.551017
Train Epoch: 27 - Loss: 0.5287, Accuracy: 99.52%

Test set: Average loss: 0.5304, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.521569
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.530898
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.529674
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.533224
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.530734
Train Epoch: 28 - Loss: 0.5279, Accuracy: 99.55%

Test set: Average loss: 0.5323, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.551244
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.520513
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.521666
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.522117
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.518089
Train Epoch: 29 - Loss: 0.5268, Accuracy: 99.56%

Test set: Average loss: 0.5274, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.520352
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.529506
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.525178
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.520742
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.549119
Train Epoch: 30 - Loss: 0.5258, Accuracy: 99.56%

Test set: Average loss: 0.5283, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.532408
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.522531
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.520963
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.514170
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.540162
Train Epoch: 31 - Loss: 0.5246, Accuracy: 99.61%

Test set: Average loss: 0.5268, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.523651
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.526996
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.546328
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.516408
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.519013
Train Epoch: 32 - Loss: 0.5238, Accuracy: 99.61%

Test set: Average loss: 0.5263, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.525129
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.530192
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.517669
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.527248
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.534374
Train Epoch: 33 - Loss: 0.5231, Accuracy: 99.64%

Test set: Average loss: 0.5264, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.514582
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.539749
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.520377
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.512263
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.518982
Train Epoch: 34 - Loss: 0.5223, Accuracy: 99.66%

Test set: Average loss: 0.5253, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.550164
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.522309
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.518411
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.515518
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.520813
Train Epoch: 35 - Loss: 0.5220, Accuracy: 99.69%

Test set: Average loss: 0.5276, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.511216
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.526164
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.516300
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.530624
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.529249
Train Epoch: 36 - Loss: 0.5214, Accuracy: 99.68%

Test set: Average loss: 0.5255, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.514832
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.516087
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.518889
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.517499
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.515934
Train Epoch: 37 - Loss: 0.5206, Accuracy: 99.72%

Test set: Average loss: 0.5241, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.524303
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.513008
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.549162
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.542914
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.524097
Train Epoch: 38 - Loss: 0.5202, Accuracy: 99.72%

Test set: Average loss: 0.5241, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.509550
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.521440
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.519747
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.514177
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.518378
Train Epoch: 39 - Loss: 0.5200, Accuracy: 99.72%

Test set: Average loss: 0.5244, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.516075
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.512529
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.511022
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.514962
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.539963
Train Epoch: 40 - Loss: 0.5196, Accuracy: 99.74%

Test set: Average loss: 0.5238, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.535800
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.511240
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.512165
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.516994
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.511922
Train Epoch: 41 - Loss: 0.5192, Accuracy: 99.73%

Test set: Average loss: 0.5235, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.523558
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.513649
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.511287
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.515231
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.527469
Train Epoch: 42 - Loss: 0.5191, Accuracy: 99.76%

Test set: Average loss: 0.5237, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.514474
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.513232
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.522453
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.512233
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.524345
Train Epoch: 43 - Loss: 0.5188, Accuracy: 99.77%

Test set: Average loss: 0.5238, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.518021
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.514275
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.522442
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.519844
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.512773
Train Epoch: 44 - Loss: 0.5187, Accuracy: 99.78%

Test set: Average loss: 0.5235, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.513822
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.515533
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.514083
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.518345
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.515696
Train Epoch: 45 - Loss: 0.5182, Accuracy: 99.78%

Test set: Average loss: 0.5231, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.522978
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.516288
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.516612
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.510077
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.547300
Train Epoch: 46 - Loss: 0.5183, Accuracy: 99.78%

Test set: Average loss: 0.5231, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.522524
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.511718
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.513931
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.508614
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.518518
Train Epoch: 47 - Loss: 0.5178, Accuracy: 99.81%

Test set: Average loss: 0.5231, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.524069
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.515497
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.521100
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.520180
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.516329
Train Epoch: 48 - Loss: 0.5179, Accuracy: 99.80%

Test set: Average loss: 0.5230, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.514152
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.526947
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.517458
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.519656
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.511657
Train Epoch: 49 - Loss: 0.5179, Accuracy: 99.80%

Test set: Average loss: 0.5228, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.515593
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.508254
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.516404
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.511900
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.522507
Train Epoch: 50 - Loss: 0.5178, Accuracy: 99.81%

Test set: Average loss: 0.5231, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.508898
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.527625
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.522290
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.540935
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.517902
Train Epoch: 51 - Loss: 0.5201, Accuracy: 99.72%

Test set: Average loss: 0.5237, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.517302
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.514407
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.516178
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.529911
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.517714
Train Epoch: 52 - Loss: 0.5195, Accuracy: 99.73%

Test set: Average loss: 0.5233, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.521940
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.516779
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.508883
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.511932
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.520163
Train Epoch: 53 - Loss: 0.5188, Accuracy: 99.74%

Test set: Average loss: 0.5236, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.518644
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.522735
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.513760
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.513370
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.512446
Train Epoch: 54 - Loss: 0.5184, Accuracy: 99.75%

Test set: Average loss: 0.5230, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.512625
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.519846
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.523156
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.512818
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.512094
Train Epoch: 55 - Loss: 0.5182, Accuracy: 99.77%

Test set: Average loss: 0.5231, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.545854
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.512163
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514003
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.519490
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.553758
Train Epoch: 56 - Loss: 0.5178, Accuracy: 99.75%

Test set: Average loss: 0.5224, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.515402
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.525969
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.534019
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.516644
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.514963
Train Epoch: 57 - Loss: 0.5174, Accuracy: 99.78%

Test set: Average loss: 0.5226, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.516470
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.516920
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.512432
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.515995
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.511816
Train Epoch: 58 - Loss: 0.5166, Accuracy: 99.81%

Test set: Average loss: 0.5225, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.518911
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.524868
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.507581
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.509409
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.535787
Train Epoch: 59 - Loss: 0.5167, Accuracy: 99.79%

Test set: Average loss: 0.5225, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.518633
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.530160
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.513167
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.515203
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.522527
Train Epoch: 60 - Loss: 0.5163, Accuracy: 99.79%

Test set: Average loss: 0.5230, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.519185
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.531533
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.513714
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.515745
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.513779
Train Epoch: 61 - Loss: 0.5156, Accuracy: 99.83%

Test set: Average loss: 0.5217, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.509693
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.517688
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.523451
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.514150
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.511840
Train Epoch: 62 - Loss: 0.5156, Accuracy: 99.82%

Test set: Average loss: 0.5214, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.510558
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.512442
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.512834
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.513128
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.510817
Train Epoch: 63 - Loss: 0.5152, Accuracy: 99.86%

Test set: Average loss: 0.5220, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.513375
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.509745
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.511553
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.511680
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.509392
Train Epoch: 64 - Loss: 0.5147, Accuracy: 99.87%

Test set: Average loss: 0.5213, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.517039
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.510197
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.516125
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.510324
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.514588
Train Epoch: 65 - Loss: 0.5146, Accuracy: 99.83%

Test set: Average loss: 0.5213, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.514730
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.509709
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.508312
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.514197
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.509821
Train Epoch: 66 - Loss: 0.5144, Accuracy: 99.87%

Test set: Average loss: 0.5212, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.525016
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.513209
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.514168
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.516480
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.509352
Train Epoch: 67 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5211, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.509070
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.513856
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.510550
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.516057
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.513476
Train Epoch: 68 - Loss: 0.5138, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.515466
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.513988
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.507029
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.511240
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.516370
Train Epoch: 69 - Loss: 0.5141, Accuracy: 99.88%

Test set: Average loss: 0.5212, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.511832
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.509863
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.518035
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.512434
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.511347
Train Epoch: 70 - Loss: 0.5140, Accuracy: 99.88%

Test set: Average loss: 0.5210, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.511273
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.530506
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.518764
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.509774
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.515242
Train Epoch: 71 - Loss: 0.5139, Accuracy: 99.87%

Test set: Average loss: 0.5210, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.517878
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.520861
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.511449
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.515538
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.509637
Train Epoch: 72 - Loss: 0.5138, Accuracy: 99.89%

Test set: Average loss: 0.5207, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.511481
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.508794
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.512966
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.509411
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.515746
Train Epoch: 73 - Loss: 0.5139, Accuracy: 99.87%

Test set: Average loss: 0.5208, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.510536
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.511599
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.513809
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.507301
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.511465
Train Epoch: 74 - Loss: 0.5136, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.512594
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.531448
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.514516
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.513324
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.514803
Train Epoch: 75 - Loss: 0.5134, Accuracy: 99.89%

Test set: Average loss: 0.5209, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.509427
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.538065
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.520031
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.513989
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.514380
Train Epoch: 76 - Loss: 0.5153, Accuracy: 99.83%

Test set: Average loss: 0.5215, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.520009
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.512519
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.513420
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.517766
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.518194
Train Epoch: 77 - Loss: 0.5151, Accuracy: 99.85%

Test set: Average loss: 0.5214, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.519547
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.509433
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.508249
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.514149
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.517982
Train Epoch: 78 - Loss: 0.5147, Accuracy: 99.86%

Test set: Average loss: 0.5211, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.510920
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.512533
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.512239
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.511578
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.513097
Train Epoch: 79 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5215, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.510168
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.512652
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.510405
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.510649
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.509050
Train Epoch: 80 - Loss: 0.5142, Accuracy: 99.87%

Test set: Average loss: 0.5239, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.507101
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.516580
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.512483
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.509122
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.514160
Train Epoch: 81 - Loss: 0.5140, Accuracy: 99.87%

Test set: Average loss: 0.5226, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.515858
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.515229
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.511038
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.515288
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.515943
Train Epoch: 82 - Loss: 0.5139, Accuracy: 99.86%

Test set: Average loss: 0.5207, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.509621
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512619
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.515180
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.509681
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.507879
Train Epoch: 83 - Loss: 0.5133, Accuracy: 99.88%

Test set: Average loss: 0.5215, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.512560
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.518248
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.519145
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.510986
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.515803
Train Epoch: 84 - Loss: 0.5132, Accuracy: 99.89%

Test set: Average loss: 0.5225, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.507132
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.513064
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.512077
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.515772
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.511862
Train Epoch: 85 - Loss: 0.5129, Accuracy: 99.89%

Test set: Average loss: 0.5214, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.512253
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.519373
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.511555
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.512602
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.518578
Train Epoch: 86 - Loss: 0.5126, Accuracy: 99.90%

Test set: Average loss: 0.5217, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.508106
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.519427
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.516723
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.516342
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.508874
Train Epoch: 87 - Loss: 0.5125, Accuracy: 99.89%

Test set: Average loss: 0.5205, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.511884
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.509948
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.511310
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.510395
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.509235
Train Epoch: 88 - Loss: 0.5123, Accuracy: 99.92%

Test set: Average loss: 0.5204, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.511017
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.513164
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.510654
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.510300
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.508106
Train Epoch: 89 - Loss: 0.5119, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.507192
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.510150
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.513749
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508471
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.515986
Train Epoch: 90 - Loss: 0.5118, Accuracy: 99.91%

Test set: Average loss: 0.5200, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.507022
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.513692
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.511849
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.511862
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.511630
Train Epoch: 91 - Loss: 0.5118, Accuracy: 99.91%

Test set: Average loss: 0.5205, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.511771
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.519029
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.513793
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.509433
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.507741
Train Epoch: 92 - Loss: 0.5116, Accuracy: 99.91%

Test set: Average loss: 0.5201, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.509832
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.505967
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.509337
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.511689
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.506298
Train Epoch: 93 - Loss: 0.5113, Accuracy: 99.94%

Test set: Average loss: 0.5198, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.521918
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.505708
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.520793
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.513045
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.511426
Train Epoch: 94 - Loss: 0.5114, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.519078
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.514058
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.508831
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.505945
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.508386
Train Epoch: 95 - Loss: 0.5113, Accuracy: 99.92%

Test set: Average loss: 0.5199, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.505448
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508383
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.510705
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.511053
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.510666
Train Epoch: 96 - Loss: 0.5115, Accuracy: 99.90%

Test set: Average loss: 0.5200, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.508628
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.509657
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.509087
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.509062
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.514166
Train Epoch: 97 - Loss: 0.5111, Accuracy: 99.92%

Test set: Average loss: 0.5198, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.508655
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.509000
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508993
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.511758
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.509060
Train Epoch: 98 - Loss: 0.5111, Accuracy: 99.93%

Test set: Average loss: 0.5197, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.508911
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.507109
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.514784
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.506755
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.506319
Train Epoch: 99 - Loss: 0.5110, Accuracy: 99.94%

Test set: Average loss: 0.5196, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.513930
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.506731
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.511342
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.509378
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.506263
Train Epoch: 100 - Loss: 0.5111, Accuracy: 99.94%

Test set: Average loss: 0.5197, Accuracy: 9938/10000 (99.38%)


Training completed. Best Accuracy: 99.41%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.329377
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.734755
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.708928
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.619017
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.590320
Train Epoch: 1 - Loss: 0.7327, Accuracy: 94.00%

Test set: Average loss: 0.6072, Accuracy: 9749/10000 (97.49%)

New best model saved with accuracy: 97.49%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.602722
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.612334
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.606320
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.607703
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.610441
Train Epoch: 2 - Loss: 0.5926, Accuracy: 98.02%

Test set: Average loss: 0.5763, Accuracy: 9837/10000 (98.37%)

New best model saved with accuracy: 98.37%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.589834
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.563643
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.560913
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.554673
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.581655
Train Epoch: 3 - Loss: 0.5707, Accuracy: 98.56%

Test set: Average loss: 0.5609, Accuracy: 9867/10000 (98.67%)

New best model saved with accuracy: 98.67%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.570466
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.573570
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.575551
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.540058
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.555997
Train Epoch: 4 - Loss: 0.5598, Accuracy: 98.81%

Test set: Average loss: 0.5529, Accuracy: 9887/10000 (98.87%)

New best model saved with accuracy: 98.87%
Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.545888
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.570683
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.543370
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.552398
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.570786
Train Epoch: 5 - Loss: 0.5523, Accuracy: 99.06%

Test set: Average loss: 0.5477, Accuracy: 9894/10000 (98.94%)

New best model saved with accuracy: 98.94%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.539912
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.543588
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.526953
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.540553
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.560949
Train Epoch: 6 - Loss: 0.5475, Accuracy: 99.15%

Test set: Average loss: 0.5427, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.546335
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.526000
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.544429
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.526979
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.542778
Train Epoch: 7 - Loss: 0.5434, Accuracy: 99.21%

Test set: Average loss: 0.5443, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.532416
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.539666
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.564078
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.521928
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.550949
Train Epoch: 8 - Loss: 0.5405, Accuracy: 99.28%

Test set: Average loss: 0.5384, Accuracy: 9912/10000 (99.12%)

New best model saved with accuracy: 99.12%
Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.521738
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.546721
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.552803
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.528935
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.540229
Train Epoch: 9 - Loss: 0.5390, Accuracy: 99.31%

Test set: Average loss: 0.5366, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.527496
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.531061
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.531085
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.532912
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.535207
Train Epoch: 10 - Loss: 0.5364, Accuracy: 99.36%

Test set: Average loss: 0.5361, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.540836
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.532128
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.531781
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.546372
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.535006
Train Epoch: 11 - Loss: 0.5351, Accuracy: 99.39%

Test set: Average loss: 0.5347, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.532010
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.558943
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.541730
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.536669
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.582284
Train Epoch: 12 - Loss: 0.5337, Accuracy: 99.48%

Test set: Average loss: 0.5336, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.522296
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.530654
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.519751
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.532603
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.542657
Train Epoch: 13 - Loss: 0.5327, Accuracy: 99.48%

Test set: Average loss: 0.5326, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.535744
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.523063
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.517107
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.527708
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.541353
Train Epoch: 14 - Loss: 0.5317, Accuracy: 99.50%

Test set: Average loss: 0.5325, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.531235
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.523643
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.526190
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.533058
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.532126
Train Epoch: 15 - Loss: 0.5307, Accuracy: 99.53%

Test set: Average loss: 0.5319, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.542086
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.541802
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.531209
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.545222
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.528264
Train Epoch: 16 - Loss: 0.5300, Accuracy: 99.53%

Test set: Average loss: 0.5309, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.531128
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.536889
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.518741
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.518173
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.542484
Train Epoch: 17 - Loss: 0.5298, Accuracy: 99.53%

Test set: Average loss: 0.5311, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.526562
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.515127
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.514447
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.526096
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.522430
Train Epoch: 18 - Loss: 0.5288, Accuracy: 99.60%

Test set: Average loss: 0.5313, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.527113
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.526859
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.536610
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.519287
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.517628
Train Epoch: 19 - Loss: 0.5285, Accuracy: 99.57%

Test set: Average loss: 0.5299, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.521213
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.523323
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.523020
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.543321
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.533759
Train Epoch: 20 - Loss: 0.5282, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.523758
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.539537
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.540951
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.528419
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.517254
Train Epoch: 21 - Loss: 0.5279, Accuracy: 99.63%

Test set: Average loss: 0.5298, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.543861
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.533751
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.520686
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.549821
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.524705
Train Epoch: 22 - Loss: 0.5278, Accuracy: 99.59%

Test set: Average loss: 0.5297, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.521961
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.523382
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.523750
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.520894
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.518574
Train Epoch: 23 - Loss: 0.5273, Accuracy: 99.62%

Test set: Average loss: 0.5298, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.529811
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.513561
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.541319
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.530121
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.527507
Train Epoch: 24 - Loss: 0.5274, Accuracy: 99.61%

Test set: Average loss: 0.5299, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.526439
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.523780
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.527795
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.527179
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.521219
Train Epoch: 25 - Loss: 0.5274, Accuracy: 99.63%

Test set: Average loss: 0.5297, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.517523
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.519909
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.526810
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.532256
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.531393
Train Epoch: 26 - Loss: 0.5298, Accuracy: 99.51%

Test set: Average loss: 0.5312, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.537652
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.524874
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.515280
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.560946
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.520597
Train Epoch: 27 - Loss: 0.5284, Accuracy: 99.58%

Test set: Average loss: 0.5338, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.538447
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.529627
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.515199
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.533869
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.540503
Train Epoch: 28 - Loss: 0.5276, Accuracy: 99.57%

Test set: Average loss: 0.5290, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.519382
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.518176
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.527403
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.523633
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.521137
Train Epoch: 29 - Loss: 0.5270, Accuracy: 99.59%

Test set: Average loss: 0.5300, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.513178
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.514137
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.517584
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.558617
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.530431
Train Epoch: 30 - Loss: 0.5257, Accuracy: 99.62%

Test set: Average loss: 0.5316, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.570644
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.531086
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.532781
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.525305
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.515863
Train Epoch: 31 - Loss: 0.5251, Accuracy: 99.58%

Test set: Average loss: 0.5300, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.519528
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.522163
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.526197
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.516909
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.539489
Train Epoch: 32 - Loss: 0.5244, Accuracy: 99.64%

Test set: Average loss: 0.5272, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.527032
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.536268
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.530416
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.524766
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.525296
Train Epoch: 33 - Loss: 0.5238, Accuracy: 99.68%

Test set: Average loss: 0.5267, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.521703
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.522761
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.536398
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.514366
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.522159
Train Epoch: 34 - Loss: 0.5230, Accuracy: 99.68%

Test set: Average loss: 0.5263, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.539490
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.517522
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.519200
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.518665
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.515146
Train Epoch: 35 - Loss: 0.5224, Accuracy: 99.69%

Test set: Average loss: 0.5252, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.533759
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.520070
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.516311
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.515294
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.528988
Train Epoch: 36 - Loss: 0.5218, Accuracy: 99.69%

Test set: Average loss: 0.5268, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.518840
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.517158
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.525731
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.514147
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.514115
Train Epoch: 37 - Loss: 0.5213, Accuracy: 99.72%

Test set: Average loss: 0.5291, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.523526
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.519164
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.525001
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.528010
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.519871
Train Epoch: 38 - Loss: 0.5209, Accuracy: 99.75%

Test set: Average loss: 0.5247, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.522249
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.536037
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.512265
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.520226
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.516346
Train Epoch: 39 - Loss: 0.5201, Accuracy: 99.79%

Test set: Average loss: 0.5245, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.510296
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.518144
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.525836
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.524283
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.512469
Train Epoch: 40 - Loss: 0.5202, Accuracy: 99.74%

Test set: Average loss: 0.5249, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.525040
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.519412
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.513937
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.530344
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.512642
Train Epoch: 41 - Loss: 0.5193, Accuracy: 99.80%

Test set: Average loss: 0.5241, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.513927
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.521620
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.513623
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.514943
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.524667
Train Epoch: 42 - Loss: 0.5192, Accuracy: 99.77%

Test set: Average loss: 0.5240, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.518702
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.517124
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.516353
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.514332
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.534220
Train Epoch: 43 - Loss: 0.5191, Accuracy: 99.79%

Test set: Average loss: 0.5238, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.511570
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.513035
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.513865
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.513398
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.514554
Train Epoch: 44 - Loss: 0.5188, Accuracy: 99.80%

Test set: Average loss: 0.5237, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.519612
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.529572
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.513676
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.521673
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.522212
Train Epoch: 45 - Loss: 0.5186, Accuracy: 99.81%

Test set: Average loss: 0.5239, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.519156
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.515516
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.513819
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.526712
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.518650
Train Epoch: 46 - Loss: 0.5184, Accuracy: 99.82%

Test set: Average loss: 0.5236, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.526132
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.514094
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.520446
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.522051
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.520172
Train Epoch: 47 - Loss: 0.5188, Accuracy: 99.79%

Test set: Average loss: 0.5239, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.512720
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.524004
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.515607
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.517772
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.512754
Train Epoch: 48 - Loss: 0.5185, Accuracy: 99.81%

Test set: Average loss: 0.5237, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.520753
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.519035
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.524511
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.514488
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.520949
Train Epoch: 49 - Loss: 0.5185, Accuracy: 99.81%

Test set: Average loss: 0.5237, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.517019
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.521236
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.512890
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.512171
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.516749
Train Epoch: 50 - Loss: 0.5181, Accuracy: 99.83%

Test set: Average loss: 0.5238, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.522147
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.523739
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.519837
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.527789
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.514669
Train Epoch: 51 - Loss: 0.5202, Accuracy: 99.72%

Test set: Average loss: 0.5246, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.523569
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.512189
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.513662
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.515555
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.514450
Train Epoch: 52 - Loss: 0.5200, Accuracy: 99.75%

Test set: Average loss: 0.5263, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.510985
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.521063
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.515256
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.509699
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.513558
Train Epoch: 53 - Loss: 0.5196, Accuracy: 99.75%

Test set: Average loss: 0.5237, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.513139
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.531832
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.512886
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.523213
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.530854
Train Epoch: 54 - Loss: 0.5190, Accuracy: 99.76%

Test set: Average loss: 0.5260, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.523279
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.527991
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.517038
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.520177
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.519334
Train Epoch: 55 - Loss: 0.5185, Accuracy: 99.77%

Test set: Average loss: 0.5235, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.512247
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.524086
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.525328
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.519685
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.528369
Train Epoch: 56 - Loss: 0.5184, Accuracy: 99.77%

Test set: Average loss: 0.5243, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.516655
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.511533
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.523623
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.512898
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.513920
Train Epoch: 57 - Loss: 0.5176, Accuracy: 99.80%

Test set: Average loss: 0.5241, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.512267
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.558552
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.528167
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.515531
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.510726
Train Epoch: 58 - Loss: 0.5174, Accuracy: 99.82%

Test set: Average loss: 0.5226, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.514152
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.517155
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.515606
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.512060
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.514429
Train Epoch: 59 - Loss: 0.5167, Accuracy: 99.82%

Test set: Average loss: 0.5224, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.509316
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.521418
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.516444
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.530600
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.510524
Train Epoch: 60 - Loss: 0.5167, Accuracy: 99.85%

Test set: Average loss: 0.5233, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.510940
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.510896
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.515411
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.518588
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.516556
Train Epoch: 61 - Loss: 0.5161, Accuracy: 99.84%

Test set: Average loss: 0.5216, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.519795
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.519470
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.512056
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.513003
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.510813
Train Epoch: 62 - Loss: 0.5159, Accuracy: 99.85%

Test set: Average loss: 0.5221, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.517446
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.512222
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.526972
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.509690
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.512727
Train Epoch: 63 - Loss: 0.5158, Accuracy: 99.83%

Test set: Average loss: 0.5219, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.512996
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.515891
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.523408
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.518777
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.523929
Train Epoch: 64 - Loss: 0.5155, Accuracy: 99.86%

Test set: Average loss: 0.5225, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.512296
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.511186
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.513864
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.522932
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.512399
Train Epoch: 65 - Loss: 0.5154, Accuracy: 99.85%

Test set: Average loss: 0.5222, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.520234
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.517248
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.528157
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.510914
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.514450
Train Epoch: 66 - Loss: 0.5149, Accuracy: 99.87%

Test set: Average loss: 0.5217, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.514569
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.516272
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.510173
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.523430
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.513408
Train Epoch: 67 - Loss: 0.5149, Accuracy: 99.85%

Test set: Average loss: 0.5213, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.511907
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.509789
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.522926
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.515324
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.515070
Train Epoch: 68 - Loss: 0.5147, Accuracy: 99.87%

Test set: Average loss: 0.5214, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.514341
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.518257
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.516057
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.515057
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.516132
Train Epoch: 69 - Loss: 0.5145, Accuracy: 99.88%

Test set: Average loss: 0.5212, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.512566
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.510894
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.519722
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.512057
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.513250
Train Epoch: 70 - Loss: 0.5146, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.514101
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.532011
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.512703
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.516155
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.512555
Train Epoch: 71 - Loss: 0.5144, Accuracy: 99.89%

Test set: Average loss: 0.5212, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.517881
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.516804
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.510075
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.511727
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.522026
Train Epoch: 72 - Loss: 0.5142, Accuracy: 99.88%

Test set: Average loss: 0.5213, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.536168
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.525382
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.517500
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.512388
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.518641
Train Epoch: 73 - Loss: 0.5144, Accuracy: 99.87%

Test set: Average loss: 0.5211, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.517647
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.516397
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.512450
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.510443
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.509941
Train Epoch: 74 - Loss: 0.5143, Accuracy: 99.88%

Test set: Average loss: 0.5213, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.511042
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.514779
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.514012
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.511755
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.519266
Train Epoch: 75 - Loss: 0.5140, Accuracy: 99.90%

Test set: Average loss: 0.5212, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.522322
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.517884
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.509727
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.515897
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.521758
Train Epoch: 76 - Loss: 0.5155, Accuracy: 99.85%

Test set: Average loss: 0.5224, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.526612
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.512228
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.509226
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.511638
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.533300
Train Epoch: 77 - Loss: 0.5156, Accuracy: 99.83%

Test set: Average loss: 0.5220, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.510997
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.513197
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.519836
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.513709
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.514156
Train Epoch: 78 - Loss: 0.5151, Accuracy: 99.84%

Test set: Average loss: 0.5215, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.511939
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.520763
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.524122
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.514104
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.512290
Train Epoch: 79 - Loss: 0.5152, Accuracy: 99.83%

Test set: Average loss: 0.5204, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.511537
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.507884
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.514862
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.510086
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.508838
Train Epoch: 80 - Loss: 0.5147, Accuracy: 99.86%

Test set: Average loss: 0.5225, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.510787
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.515226
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.514912
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.510544
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.512218
Train Epoch: 81 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5224, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.508951
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.514431
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.511881
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.512659
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.511441
Train Epoch: 82 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5219, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.519041
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.522098
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.509290
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.511482
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.513980
Train Epoch: 83 - Loss: 0.5138, Accuracy: 99.88%

Test set: Average loss: 0.5212, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.514191
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.515331
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.507682
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.509985
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.510771
Train Epoch: 84 - Loss: 0.5137, Accuracy: 99.89%

Test set: Average loss: 0.5210, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.510206
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.518376
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.521421
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.511598
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.515766
Train Epoch: 85 - Loss: 0.5132, Accuracy: 99.89%

Test set: Average loss: 0.5209, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.514447
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.508465
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.508657
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.510277
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.511165
Train Epoch: 86 - Loss: 0.5132, Accuracy: 99.89%

Test set: Average loss: 0.5214, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.519040
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.510278
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.515472
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.510119
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.509834
Train Epoch: 87 - Loss: 0.5129, Accuracy: 99.90%

Test set: Average loss: 0.5207, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.513180
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.512261
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.514255
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.510305
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.510307
Train Epoch: 88 - Loss: 0.5130, Accuracy: 99.90%

Test set: Average loss: 0.5202, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.515546
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.511563
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.514466
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.510804
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.518212
Train Epoch: 89 - Loss: 0.5126, Accuracy: 99.89%

Test set: Average loss: 0.5200, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.510281
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.513140
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.512037
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.510656
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.519540
Train Epoch: 90 - Loss: 0.5125, Accuracy: 99.91%

Test set: Average loss: 0.5210, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.509520
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.506895
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.515585
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.512770
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.509849
Train Epoch: 91 - Loss: 0.5124, Accuracy: 99.90%

Test set: Average loss: 0.5201, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.507335
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510160
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.514108
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.507052
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.508829
Train Epoch: 92 - Loss: 0.5123, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.513107
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.512881
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.509804
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.519913
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.511213
Train Epoch: 93 - Loss: 0.5120, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.513886
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.517491
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.510036
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.513446
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.508052
Train Epoch: 94 - Loss: 0.5119, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.512062
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.508594
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.512362
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.509232
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.510033
Train Epoch: 95 - Loss: 0.5117, Accuracy: 99.93%

Test set: Average loss: 0.5200, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.508550
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.522508
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.507608
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.510128
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.509033
Train Epoch: 96 - Loss: 0.5119, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.507631
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.506719
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.515591
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.514394
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.516929
Train Epoch: 97 - Loss: 0.5118, Accuracy: 99.93%

Test set: Average loss: 0.5201, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.513656
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.510626
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.513431
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.514770
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.517914
Train Epoch: 98 - Loss: 0.5118, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.513948
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.529033
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.509760
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.510073
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.508448
Train Epoch: 99 - Loss: 0.5118, Accuracy: 99.92%

Test set: Average loss: 0.5198, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.515455
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.517373
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.507164
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.507945
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.514450
Train Epoch: 100 - Loss: 0.5117, Accuracy: 99.93%

Test set: Average loss: 0.5199, Accuracy: 9937/10000 (99.37%)


Training completed. Best Accuracy: 99.43%
Current learning rate: 0.100000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.372285
Train Epoch: 1 [12800/60000 (21%)]	Loss: 0.761525
Train Epoch: 1 [25600/60000 (43%)]	Loss: 0.667792
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.628379
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.590217
Train Epoch: 1 - Loss: 0.7125, Accuracy: 94.38%

Test set: Average loss: 0.5899, Accuracy: 9799/10000 (97.99%)

New best model saved with accuracy: 97.99%
Current learning rate: 0.099606
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.631266
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.600890
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.567364
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.564723
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.593356
Train Epoch: 2 - Loss: 0.5851, Accuracy: 98.16%

Test set: Average loss: 0.5779, Accuracy: 9831/10000 (98.31%)

New best model saved with accuracy: 98.31%
Current learning rate: 0.098429
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.569751
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.601197
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.556705
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.584501
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.565478
Train Epoch: 3 - Loss: 0.5669, Accuracy: 98.62%

Test set: Average loss: 0.5568, Accuracy: 9880/10000 (98.80%)

New best model saved with accuracy: 98.80%
Current learning rate: 0.096489
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.539580
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.573437
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.544680
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.580090
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.556109
Train Epoch: 4 - Loss: 0.5567, Accuracy: 98.88%

Test set: Average loss: 0.5551, Accuracy: 9879/10000 (98.79%)

Current learning rate: 0.093815
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.540068
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.539473
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.570569
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.575099
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.556210
Train Epoch: 5 - Loss: 0.5505, Accuracy: 99.03%

Test set: Average loss: 0.5505, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.090451
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.541011
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.549631
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.535565
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.525215
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.541195
Train Epoch: 6 - Loss: 0.5459, Accuracy: 99.13%

Test set: Average loss: 0.5429, Accuracy: 9908/10000 (99.08%)

New best model saved with accuracy: 99.08%
Current learning rate: 0.086448
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.527736
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.558134
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.548223
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.550015
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.529929
Train Epoch: 7 - Loss: 0.5423, Accuracy: 99.27%

Test set: Average loss: 0.5391, Accuracy: 9915/10000 (99.15%)

New best model saved with accuracy: 99.15%
Current learning rate: 0.081871
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.535385
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.542477
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.540789
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.533082
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.535686
Train Epoch: 8 - Loss: 0.5399, Accuracy: 99.27%

Test set: Average loss: 0.5417, Accuracy: 9904/10000 (99.04%)

Current learning rate: 0.076791
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.529412
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.550521
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.541385
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.550380
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.529227
Train Epoch: 9 - Loss: 0.5375, Accuracy: 99.31%

Test set: Average loss: 0.5374, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.071289
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.546348
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.539815
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.528621
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.527378
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.536850
Train Epoch: 10 - Loss: 0.5358, Accuracy: 99.36%

Test set: Average loss: 0.5351, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.065451
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.519762
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.523210
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.537482
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.532040
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.525771
Train Epoch: 11 - Loss: 0.5340, Accuracy: 99.46%

Test set: Average loss: 0.5339, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.059369
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.529698
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.526274
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.528716
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.523969
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.528927
Train Epoch: 12 - Loss: 0.5328, Accuracy: 99.45%

Test set: Average loss: 0.5343, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.053140
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.537445
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.525637
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.540972
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.515978
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.528387
Train Epoch: 13 - Loss: 0.5320, Accuracy: 99.47%

Test set: Average loss: 0.5324, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.046860
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.531588
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.577041
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.534806
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.547923
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.528638
Train Epoch: 14 - Loss: 0.5311, Accuracy: 99.48%

Test set: Average loss: 0.5319, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.040631
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.524938
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.536109
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.556792
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.524598
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.531175
Train Epoch: 15 - Loss: 0.5303, Accuracy: 99.53%

Test set: Average loss: 0.5319, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.034549
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.536041
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.522178
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.519456
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.523564
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.539071
Train Epoch: 16 - Loss: 0.5295, Accuracy: 99.50%

Test set: Average loss: 0.5308, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.028711
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.546051
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.538886
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.528699
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.530937
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.540382
Train Epoch: 17 - Loss: 0.5288, Accuracy: 99.54%

Test set: Average loss: 0.5311, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.023209
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.535686
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.558208
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.522164
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.536356
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.526752
Train Epoch: 18 - Loss: 0.5285, Accuracy: 99.56%

Test set: Average loss: 0.5305, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.018129
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.515718
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.522731
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.536307
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.537105
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.526700
Train Epoch: 19 - Loss: 0.5279, Accuracy: 99.58%

Test set: Average loss: 0.5301, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.013552
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.524375
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.518212
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.521410
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.527220
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.517733
Train Epoch: 20 - Loss: 0.5280, Accuracy: 99.58%

Test set: Average loss: 0.5302, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.009549
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.515272
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.519579
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.534958
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.530655
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.516039
Train Epoch: 21 - Loss: 0.5272, Accuracy: 99.60%

Test set: Average loss: 0.5301, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.006185
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.525959
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.520936
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.519589
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.545198
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.516214
Train Epoch: 22 - Loss: 0.5273, Accuracy: 99.59%

Test set: Average loss: 0.5298, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.003511
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.539152
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.516664
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.534331
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.527699
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.521111
Train Epoch: 23 - Loss: 0.5274, Accuracy: 99.59%

Test set: Average loss: 0.5298, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.001571
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.519056
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.527445
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.527334
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.528154
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.523176
Train Epoch: 24 - Loss: 0.5274, Accuracy: 99.61%

Test set: Average loss: 0.5299, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000394
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.519248
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.541625
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.540585
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.532055
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.538010
Train Epoch: 25 - Loss: 0.5272, Accuracy: 99.61%

Test set: Average loss: 0.5297, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.100000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.529562
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.524175
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.543025
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.528037
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.536700
Train Epoch: 26 - Loss: 0.5297, Accuracy: 99.51%

Test set: Average loss: 0.5306, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.099606
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.533941
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.518244
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.525355
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.520755
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.517047
Train Epoch: 27 - Loss: 0.5283, Accuracy: 99.56%

Test set: Average loss: 0.5300, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.098429
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.519337
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.522729
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.527913
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.533164
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.524124
Train Epoch: 28 - Loss: 0.5273, Accuracy: 99.59%

Test set: Average loss: 0.5296, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.096489
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.515104
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.522343
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.525365
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.523354
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.525634
Train Epoch: 29 - Loss: 0.5265, Accuracy: 99.58%

Test set: Average loss: 0.5297, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.093815
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.526096
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.520309
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.523277
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.524611
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.534643
Train Epoch: 30 - Loss: 0.5256, Accuracy: 99.59%

Test set: Average loss: 0.5272, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.090451
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.518961
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.540628
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.537633
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.524815
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.528137
Train Epoch: 31 - Loss: 0.5247, Accuracy: 99.60%

Test set: Average loss: 0.5312, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.086448
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.516656
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.510118
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.523348
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.515794
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.516395
Train Epoch: 32 - Loss: 0.5239, Accuracy: 99.64%

Test set: Average loss: 0.5278, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.081871
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.518499
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.517950
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.515361
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.513364
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.519536
Train Epoch: 33 - Loss: 0.5230, Accuracy: 99.67%

Test set: Average loss: 0.5266, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.076791
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.538619
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.527456
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.531471
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.514655
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.526428
Train Epoch: 34 - Loss: 0.5226, Accuracy: 99.69%

Test set: Average loss: 0.5280, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.071289
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.518244
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.517627
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.530064
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.526683
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.524725
Train Epoch: 35 - Loss: 0.5221, Accuracy: 99.68%

Test set: Average loss: 0.5256, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.065451
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.516848
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.513658
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.522733
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.525784
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.515836
Train Epoch: 36 - Loss: 0.5212, Accuracy: 99.71%

Test set: Average loss: 0.5266, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.059369
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.519544
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.518149
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.516384
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.515926
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.521787
Train Epoch: 37 - Loss: 0.5211, Accuracy: 99.73%

Test set: Average loss: 0.5256, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.053140
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.515141
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.516682
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.517609
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.511366
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.523631
Train Epoch: 38 - Loss: 0.5204, Accuracy: 99.75%

Test set: Average loss: 0.5259, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.046860
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.513501
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.514723
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.518331
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.517166
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.509256
Train Epoch: 39 - Loss: 0.5206, Accuracy: 99.72%

Test set: Average loss: 0.5248, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.040631
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.523558
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.517671
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.520625
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.525783
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.521279
Train Epoch: 40 - Loss: 0.5199, Accuracy: 99.76%

Test set: Average loss: 0.5244, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.034549
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.513999
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.521234
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.510051
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.528446
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.525154
Train Epoch: 41 - Loss: 0.5194, Accuracy: 99.77%

Test set: Average loss: 0.5244, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.028711
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.519349
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.536148
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.514188
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.519334
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.521692
Train Epoch: 42 - Loss: 0.5190, Accuracy: 99.78%

Test set: Average loss: 0.5244, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.023209
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.512676
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.517380
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.523421
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.519253
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.515617
Train Epoch: 43 - Loss: 0.5193, Accuracy: 99.77%

Test set: Average loss: 0.5242, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.018129
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.520136
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.520768
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.513523
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.519747
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.509361
Train Epoch: 44 - Loss: 0.5188, Accuracy: 99.77%

Test set: Average loss: 0.5241, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.013552
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.528371
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.526530
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.518076
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.516959
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.514156
Train Epoch: 45 - Loss: 0.5186, Accuracy: 99.79%

Test set: Average loss: 0.5239, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.009549
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.514091
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.511281
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.525701
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.516655
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.554188
Train Epoch: 46 - Loss: 0.5186, Accuracy: 99.78%

Test set: Average loss: 0.5242, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.006185
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.517236
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.513628
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.526142
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.514492
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.517487
Train Epoch: 47 - Loss: 0.5184, Accuracy: 99.79%

Test set: Average loss: 0.5240, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.003511
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.513318
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.520113
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.513395
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.513763
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.516489
Train Epoch: 48 - Loss: 0.5186, Accuracy: 99.81%

Test set: Average loss: 0.5238, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.001571
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.517592
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.524308
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.516668
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.518338
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.524618
Train Epoch: 49 - Loss: 0.5181, Accuracy: 99.80%

Test set: Average loss: 0.5240, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000394
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.526085
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.547696
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.533921
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.512367
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.514688
Train Epoch: 50 - Loss: 0.5184, Accuracy: 99.81%

Test set: Average loss: 0.5238, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.100000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.513763
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.524210
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.526756
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.524444
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.520723
Train Epoch: 51 - Loss: 0.5200, Accuracy: 99.74%

Test set: Average loss: 0.5247, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.099606
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.523184
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.527545
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.512718
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.517595
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.518418
Train Epoch: 52 - Loss: 0.5197, Accuracy: 99.74%

Test set: Average loss: 0.5256, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.098429
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.513906
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.520823
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.523016
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.519603
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.514749
Train Epoch: 53 - Loss: 0.5192, Accuracy: 99.74%

Test set: Average loss: 0.5256, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.096489
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.520315
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.516856
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.514580
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.517270
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.512033
Train Epoch: 54 - Loss: 0.5188, Accuracy: 99.75%

Test set: Average loss: 0.5261, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.093815
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.524536
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.512445
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.516284
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.536242
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.517945
Train Epoch: 55 - Loss: 0.5182, Accuracy: 99.79%

Test set: Average loss: 0.5242, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.090451
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.537013
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.518553
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.514657
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.518122
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.513739
Train Epoch: 56 - Loss: 0.5177, Accuracy: 99.77%

Test set: Average loss: 0.5242, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.086448
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.540815
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.511041
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.518502
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.513207
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.510316
Train Epoch: 57 - Loss: 0.5172, Accuracy: 99.82%

Test set: Average loss: 0.5237, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.081871
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.512034
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.515175
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.511913
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.532700
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.516985
Train Epoch: 58 - Loss: 0.5171, Accuracy: 99.80%

Test set: Average loss: 0.5239, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.076791
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.527710
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.512776
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.533238
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.511101
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.513060
Train Epoch: 59 - Loss: 0.5165, Accuracy: 99.80%

Test set: Average loss: 0.5229, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.071289
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.514953
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.510855
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.511988
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.517567
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.515294
Train Epoch: 60 - Loss: 0.5164, Accuracy: 99.81%

Test set: Average loss: 0.5215, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.065451
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.509663
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.523881
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.516047
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.513352
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.513563
Train Epoch: 61 - Loss: 0.5162, Accuracy: 99.83%

Test set: Average loss: 0.5219, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.059369
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.514589
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.513462
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.513248
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.520280
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.512336
Train Epoch: 62 - Loss: 0.5159, Accuracy: 99.83%

Test set: Average loss: 0.5216, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.053140
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.509689
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.516519
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.512723
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.511918
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.514413
Train Epoch: 63 - Loss: 0.5153, Accuracy: 99.84%

Test set: Average loss: 0.5215, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.046860
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.514463
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.509370
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.512688
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.511344
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.523654
Train Epoch: 64 - Loss: 0.5154, Accuracy: 99.84%

Test set: Average loss: 0.5217, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.040631
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.511966
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.527869
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.508152
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.528110
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.511332
Train Epoch: 65 - Loss: 0.5151, Accuracy: 99.84%

Test set: Average loss: 0.5216, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.034549
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.518429
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.518249
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.509777
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.518091
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.522336
Train Epoch: 66 - Loss: 0.5149, Accuracy: 99.85%

Test set: Average loss: 0.5211, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.028711
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.511391
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.514625
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.512778
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.510072
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.509403
Train Epoch: 67 - Loss: 0.5147, Accuracy: 99.87%

Test set: Average loss: 0.5212, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.023209
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.516643
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.510510
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.508434
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.510258
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.516977
Train Epoch: 68 - Loss: 0.5145, Accuracy: 99.87%

Test set: Average loss: 0.5212, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.018129
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.514301
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.514704
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.517946
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.510309
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.518222
Train Epoch: 69 - Loss: 0.5143, Accuracy: 99.86%

Test set: Average loss: 0.5211, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.013552
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.511483
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.513381
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.510137
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.520517
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.517552
Train Epoch: 70 - Loss: 0.5144, Accuracy: 99.86%

Test set: Average loss: 0.5213, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.009549
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.511057
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.512052
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.511019
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.510314
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.513774
Train Epoch: 71 - Loss: 0.5141, Accuracy: 99.89%

Test set: Average loss: 0.5213, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.006185
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.509732
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.509068
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.514222
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.531240
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.519121
Train Epoch: 72 - Loss: 0.5142, Accuracy: 99.87%

Test set: Average loss: 0.5211, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.003511
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.520294
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.525215
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.509284
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.544183
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.516199
Train Epoch: 73 - Loss: 0.5143, Accuracy: 99.87%

Test set: Average loss: 0.5212, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.513471
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.516994
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.515222
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.527461
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.510923
Train Epoch: 74 - Loss: 0.5140, Accuracy: 99.89%

Test set: Average loss: 0.5211, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000394
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.512349
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.529298
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.515472
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.525694
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.516367
Train Epoch: 75 - Loss: 0.5141, Accuracy: 99.90%

Test set: Average loss: 0.5212, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.100000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.509148
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.516628
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.513460
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.515832
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.526763
Train Epoch: 76 - Loss: 0.5154, Accuracy: 99.83%

Test set: Average loss: 0.5214, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.099606
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.519921
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.511557
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.526103
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.509613
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.515802
Train Epoch: 77 - Loss: 0.5151, Accuracy: 99.85%

Test set: Average loss: 0.5288, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.098429
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.513868
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.510720
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.511489
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.509672
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.513964
Train Epoch: 78 - Loss: 0.5150, Accuracy: 99.84%

Test set: Average loss: 0.5214, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.096489
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.511835
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.509080
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.511582
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.522511
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.512696
Train Epoch: 79 - Loss: 0.5146, Accuracy: 99.87%

Test set: Average loss: 0.5220, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.093815
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.512578
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.507876
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.512745
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.511627
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.521503
Train Epoch: 80 - Loss: 0.5146, Accuracy: 99.86%

Test set: Average loss: 0.5215, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.090451
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.512526
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.509342
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.536326
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.510917
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.508721
Train Epoch: 81 - Loss: 0.5142, Accuracy: 99.88%

Test set: Average loss: 0.5209, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.086448
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.510104
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.517161
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.514460
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.512048
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.510653
Train Epoch: 82 - Loss: 0.5141, Accuracy: 99.87%

Test set: Average loss: 0.5219, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.081871
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.514171
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.512131
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.519928
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.511494
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.509028
Train Epoch: 83 - Loss: 0.5137, Accuracy: 99.88%

Test set: Average loss: 0.5207, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.076791
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.509383
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.515806
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.511530
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.506545
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.523438
Train Epoch: 84 - Loss: 0.5137, Accuracy: 99.87%

Test set: Average loss: 0.5209, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.071289
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.513842
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.509193
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.511561
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.521980
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.510531
Train Epoch: 85 - Loss: 0.5134, Accuracy: 99.88%

Test set: Average loss: 0.5209, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.065451
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.511389
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.522546
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.511058
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.511105
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.512517
Train Epoch: 86 - Loss: 0.5132, Accuracy: 99.88%

Test set: Average loss: 0.5204, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.059369
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.517256
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.517323
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.508430
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.512385
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.509647
Train Epoch: 87 - Loss: 0.5127, Accuracy: 99.89%

Test set: Average loss: 0.5205, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.053140
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.521010
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.507032
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.509578
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.507087
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.516529
Train Epoch: 88 - Loss: 0.5125, Accuracy: 99.91%

Test set: Average loss: 0.5203, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.046860
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.510672
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.514455
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.511478
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.509370
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.514083
Train Epoch: 89 - Loss: 0.5123, Accuracy: 99.90%

Test set: Average loss: 0.5198, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.040631
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.509998
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.508503
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.507249
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.508347
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.509575
Train Epoch: 90 - Loss: 0.5123, Accuracy: 99.91%

Test set: Average loss: 0.5198, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.034549
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.509098
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.525591
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.518540
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.509646
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.509957
Train Epoch: 91 - Loss: 0.5121, Accuracy: 99.92%

Test set: Average loss: 0.5200, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.028711
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.509422
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.510233
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.511483
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.516565
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.521607
Train Epoch: 92 - Loss: 0.5121, Accuracy: 99.91%

Test set: Average loss: 0.5196, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.023209
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.512352
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.510626
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.508779
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.507876
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.513538
Train Epoch: 93 - Loss: 0.5119, Accuracy: 99.92%

Test set: Average loss: 0.5202, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.018129
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.509071
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.518913
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.515601
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.516208
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.511397
Train Epoch: 94 - Loss: 0.5121, Accuracy: 99.91%

Test set: Average loss: 0.5198, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.013552
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.507540
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.512391
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.506908
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.516104
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.516937
Train Epoch: 95 - Loss: 0.5117, Accuracy: 99.90%

Test set: Average loss: 0.5196, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.009549
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.514600
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.508204
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.508316
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.512132
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.509535
Train Epoch: 96 - Loss: 0.5117, Accuracy: 99.94%

Test set: Average loss: 0.5195, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.006185
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.509652
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.508948
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.514686
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.513510
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.507720
Train Epoch: 97 - Loss: 0.5116, Accuracy: 99.92%

Test set: Average loss: 0.5198, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.003511
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.508927
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.511714
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.508894
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.508215
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.512197
Train Epoch: 98 - Loss: 0.5116, Accuracy: 99.92%

Test set: Average loss: 0.5195, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.001571
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.510049
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.509254
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.506205
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.519910
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.511135
Train Epoch: 99 - Loss: 0.5116, Accuracy: 99.92%

Test set: Average loss: 0.5196, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000394
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.508546
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.510198
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.509526
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.508841
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.512002
Train Epoch: 100 - Loss: 0.5115, Accuracy: 99.92%

Test set: Average loss: 0.5194, Accuracy: 9943/10000 (99.43%)


Training completed. Best Accuracy: 99.45%
