Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.219114
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.357571
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.196815
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.168849
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.994880
Train Epoch: 1 - Loss: 1.3077, Accuracy: 70.42%

Test set: Average loss: 0.6786, Accuracy: 9666/10000 (96.66%)

New best model saved with accuracy: 96.66%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.944720
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.854537
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.909325
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.823660
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.894841
Train Epoch: 2 - Loss: 0.8675, Accuracy: 86.34%

Test set: Average loss: 0.6021, Accuracy: 9810/10000 (98.10%)

New best model saved with accuracy: 98.10%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.818956
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.807759
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.800814
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.763802
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.810269
Train Epoch: 3 - Loss: 0.8017, Accuracy: 88.57%

Test set: Average loss: 0.5752, Accuracy: 9855/10000 (98.55%)

New best model saved with accuracy: 98.55%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.823765
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.782966
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.747984
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.703538
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.798723
Train Epoch: 4 - Loss: 0.7626, Accuracy: 90.26%

Test set: Average loss: 0.5701, Accuracy: 9867/10000 (98.67%)

New best model saved with accuracy: 98.67%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.693370
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.686957
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.824358
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.790581
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.739743
Train Epoch: 5 - Loss: 0.7459, Accuracy: 90.69%

Test set: Average loss: 0.5548, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.735528
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.772584
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.776093
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.749414
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.742826
Train Epoch: 6 - Loss: 0.7301, Accuracy: 91.17%

Test set: Average loss: 0.5616, Accuracy: 9867/10000 (98.67%)

Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.724544
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.705365
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.677774
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.653467
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.734716
Train Epoch: 7 - Loss: 0.7173, Accuracy: 91.68%

Test set: Average loss: 0.5453, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.722322
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.747722
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.638001
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.814522
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.644864
Train Epoch: 8 - Loss: 0.7082, Accuracy: 91.99%

Test set: Average loss: 0.5412, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.685358
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.666366
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.690209
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.640520
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.639915
Train Epoch: 9 - Loss: 0.7011, Accuracy: 92.24%

Test set: Average loss: 0.5423, Accuracy: 9912/10000 (99.12%)

Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.636793
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.668803
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.735609
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.679793
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.652703
Train Epoch: 10 - Loss: 0.6941, Accuracy: 92.53%

Test set: Average loss: 0.5413, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.661382
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.616352
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.707260
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.718177
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.668998
Train Epoch: 11 - Loss: 0.6896, Accuracy: 92.69%

Test set: Average loss: 0.5355, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.696520
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.652011
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.631982
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.687976
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.703963
Train Epoch: 12 - Loss: 0.6858, Accuracy: 92.83%

Test set: Average loss: 0.5361, Accuracy: 9920/10000 (99.20%)

New best model saved with accuracy: 99.20%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.740908
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.684743
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.601777
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.672840
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.705615
Train Epoch: 13 - Loss: 0.6828, Accuracy: 92.97%

Test set: Average loss: 0.5345, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.674045
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.605352
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.690214
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.637989
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.725469
Train Epoch: 14 - Loss: 0.6802, Accuracy: 93.03%

Test set: Average loss: 0.5349, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.614503
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.704898
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.637117
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.615045
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.692039
Train Epoch: 15 - Loss: 0.6781, Accuracy: 93.12%

Test set: Average loss: 0.5323, Accuracy: 9923/10000 (99.23%)

New best model saved with accuracy: 99.23%
Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.614252
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.678364
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.716285
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.619369
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.736547
Train Epoch: 16 - Loss: 0.6762, Accuracy: 93.13%

Test set: Average loss: 0.5318, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.650610
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.704954
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.570646
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.653330
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.680844
Train Epoch: 17 - Loss: 0.6708, Accuracy: 93.35%

Test set: Average loss: 0.5295, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.645624
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.708924
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.655411
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.626958
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.713832
Train Epoch: 18 - Loss: 0.6680, Accuracy: 93.49%

Test set: Average loss: 0.5310, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.658459
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.604968
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.746750
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.661218
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.688032
Train Epoch: 19 - Loss: 0.6697, Accuracy: 93.49%

Test set: Average loss: 0.5301, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.666628
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.695801
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.675227
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.768572
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.624336
Train Epoch: 20 - Loss: 0.6681, Accuracy: 93.48%

Test set: Average loss: 0.5301, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.636005
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.697878
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.611305
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.640242
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.665708
Train Epoch: 21 - Loss: 0.6643, Accuracy: 93.64%

Test set: Average loss: 0.5290, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.740941
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.704337
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.665087
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.675351
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.650535
Train Epoch: 22 - Loss: 0.6658, Accuracy: 93.59%

Test set: Average loss: 0.5289, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.713993
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.635584
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.606176
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.717470
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.630016
Train Epoch: 23 - Loss: 0.6658, Accuracy: 93.48%

Test set: Average loss: 0.5284, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.652219
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.669433
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.674448
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.672749
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.666909
Train Epoch: 24 - Loss: 0.6664, Accuracy: 93.45%

Test set: Average loss: 0.5289, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.630381
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.650405
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.617503
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.718321
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.729268
Train Epoch: 25 - Loss: 0.6639, Accuracy: 93.68%

Test set: Average loss: 0.5290, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.717661
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.704980
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.713353
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.612036
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.694829
Train Epoch: 26 - Loss: 0.6750, Accuracy: 93.12%

Test set: Average loss: 0.5371, Accuracy: 9916/10000 (99.16%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.686604
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.659713
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.693188
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.718763
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.685929
Train Epoch: 27 - Loss: 0.6771, Accuracy: 93.08%

Test set: Average loss: 0.5285, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.653137
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.653763
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.677047
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.660924
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.676903
Train Epoch: 28 - Loss: 0.6713, Accuracy: 93.24%

Test set: Average loss: 0.5293, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.663702
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.631223
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.688518
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.628432
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.735155
Train Epoch: 29 - Loss: 0.6705, Accuracy: 93.41%

Test set: Average loss: 0.5288, Accuracy: 9937/10000 (99.37%)

New best model saved with accuracy: 99.37%
Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.689959
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.667235
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.653051
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.649805
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.673240
Train Epoch: 30 - Loss: 0.6676, Accuracy: 93.52%

Test set: Average loss: 0.5297, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.711893
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.701640
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.663838
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.663018
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.685720
Train Epoch: 31 - Loss: 0.6630, Accuracy: 93.51%

Test set: Average loss: 0.5264, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.654661
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.655387
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.634689
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.653910
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.652637
Train Epoch: 32 - Loss: 0.6608, Accuracy: 93.63%

Test set: Average loss: 0.5272, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.610296
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.646348
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.654320
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.675159
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.634151
Train Epoch: 33 - Loss: 0.6586, Accuracy: 93.73%

Test set: Average loss: 0.5265, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.636671
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.653023
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.658898
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.734788
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.641632
Train Epoch: 34 - Loss: 0.6574, Accuracy: 93.90%

Test set: Average loss: 0.5266, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.656714
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.621683
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.669939
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.646023
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.625929
Train Epoch: 35 - Loss: 0.6527, Accuracy: 93.94%

Test set: Average loss: 0.5246, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.621711
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.576422
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.656865
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.626408
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.674762
Train Epoch: 36 - Loss: 0.6543, Accuracy: 93.92%

Test set: Average loss: 0.5236, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.712775
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.583867
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.671153
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.595506
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.584307
Train Epoch: 37 - Loss: 0.6512, Accuracy: 94.07%

Test set: Average loss: 0.5233, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.650337
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.618276
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.629392
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.670569
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.686630
Train Epoch: 38 - Loss: 0.6500, Accuracy: 94.05%

Test set: Average loss: 0.5231, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.659154
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.652478
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.692878
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.652022
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.642960
Train Epoch: 39 - Loss: 0.6494, Accuracy: 94.04%

Test set: Average loss: 0.5241, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.773386
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.673501
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.655495
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.677683
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.684158
Train Epoch: 40 - Loss: 0.6456, Accuracy: 94.27%

Test set: Average loss: 0.5234, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.717864
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.694052
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.610339
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.726878
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.646013
Train Epoch: 41 - Loss: 0.6477, Accuracy: 94.19%

Test set: Average loss: 0.5224, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.606305
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.631071
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.620007
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.633087
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.635722
Train Epoch: 42 - Loss: 0.6440, Accuracy: 94.26%

Test set: Average loss: 0.5219, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.614538
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.615183
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.683619
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.597337
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.636080
Train Epoch: 43 - Loss: 0.6427, Accuracy: 94.32%

Test set: Average loss: 0.5220, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.651436
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.615066
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.570470
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.679708
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.592035
Train Epoch: 44 - Loss: 0.6418, Accuracy: 94.28%

Test set: Average loss: 0.5216, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.697351
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.684812
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.646140
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.697094
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.596694
Train Epoch: 45 - Loss: 0.6403, Accuracy: 94.48%

Test set: Average loss: 0.5214, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.666528
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.681759
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.586367
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.624606
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.606499
Train Epoch: 46 - Loss: 0.6425, Accuracy: 94.34%

Test set: Average loss: 0.5217, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.614708
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.630111
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.642294
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.657545
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.654612
Train Epoch: 47 - Loss: 0.6398, Accuracy: 94.46%

Test set: Average loss: 0.5222, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.720391
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.656181
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.678679
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.691299
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.660968
Train Epoch: 48 - Loss: 0.6394, Accuracy: 94.41%

Test set: Average loss: 0.5216, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.640632
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.650254
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.638096
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.701261
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.607103
Train Epoch: 49 - Loss: 0.6400, Accuracy: 94.48%

Test set: Average loss: 0.5216, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.598983
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.571723
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.654381
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.624447
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.662404
Train Epoch: 50 - Loss: 0.6409, Accuracy: 94.39%

Test set: Average loss: 0.5214, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.600157
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.656370
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.658022
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.653467
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.624288
Train Epoch: 51 - Loss: 0.6500, Accuracy: 94.07%

Test set: Average loss: 0.5244, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.623397
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.665962
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.610571
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.627670
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.652616
Train Epoch: 52 - Loss: 0.6506, Accuracy: 94.01%

Test set: Average loss: 0.5247, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.716992
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.745659
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.630043
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.634185
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.687683
Train Epoch: 53 - Loss: 0.6489, Accuracy: 94.05%

Test set: Average loss: 0.5228, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.660159
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.702510
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.581527
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.657842
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.657187
Train Epoch: 54 - Loss: 0.6479, Accuracy: 94.08%

Test set: Average loss: 0.5238, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.663490
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.685879
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.656151
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.651491
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.666945
Train Epoch: 55 - Loss: 0.6461, Accuracy: 94.15%

Test set: Average loss: 0.5223, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.669091
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.663692
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.698198
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.624463
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.644119
Train Epoch: 56 - Loss: 0.6441, Accuracy: 94.19%

Test set: Average loss: 0.5250, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.698393
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.568862
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.598979
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.676365
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.627925
Train Epoch: 57 - Loss: 0.6420, Accuracy: 94.35%

Test set: Average loss: 0.5215, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.617368
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.665651
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.683139
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.621193
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.635887
Train Epoch: 58 - Loss: 0.6429, Accuracy: 94.27%

Test set: Average loss: 0.5211, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.624013
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.639997
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.655964
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.667104
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.600116
Train Epoch: 59 - Loss: 0.6426, Accuracy: 94.23%

Test set: Average loss: 0.5203, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.592132
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.684996
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.649768
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.695344
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.628769
Train Epoch: 60 - Loss: 0.6407, Accuracy: 94.34%

Test set: Average loss: 0.5208, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.681550
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.683210
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.620403
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.610821
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.617993
Train Epoch: 61 - Loss: 0.6393, Accuracy: 94.40%

Test set: Average loss: 0.5210, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.618506
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.634193
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.656314
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.698506
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.712265
Train Epoch: 62 - Loss: 0.6377, Accuracy: 94.44%

Test set: Average loss: 0.5200, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.635894
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.624278
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.655506
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.573872
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.627817
Train Epoch: 63 - Loss: 0.6326, Accuracy: 94.61%

Test set: Average loss: 0.5188, Accuracy: 9951/10000 (99.51%)

New best model saved with accuracy: 99.51%
Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.652064
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.699466
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.566479
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.646367
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.662926
Train Epoch: 64 - Loss: 0.6363, Accuracy: 94.56%

Test set: Average loss: 0.5192, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.656021
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.607363
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.660329
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.636732
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.604361
Train Epoch: 65 - Loss: 0.6346, Accuracy: 94.63%

Test set: Average loss: 0.5187, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.672059
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.655428
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.613389
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.659933
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.713251
Train Epoch: 66 - Loss: 0.6352, Accuracy: 94.58%

Test set: Average loss: 0.5193, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.663628
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.646524
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.658049
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.679964
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.661835
Train Epoch: 67 - Loss: 0.6326, Accuracy: 94.70%

Test set: Average loss: 0.5187, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.612213
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.615026
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.612686
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.616977
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.633158
Train Epoch: 68 - Loss: 0.6308, Accuracy: 94.74%

Test set: Average loss: 0.5184, Accuracy: 9955/10000 (99.55%)

New best model saved with accuracy: 99.55%
Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.626765
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.619443
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.589391
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.631945
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.615978
Train Epoch: 69 - Loss: 0.6298, Accuracy: 94.80%

Test set: Average loss: 0.5185, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.614087
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.613999
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.584805
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.618944
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.571556
Train Epoch: 70 - Loss: 0.6310, Accuracy: 94.78%

Test set: Average loss: 0.5180, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.572843
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.581897
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.612180
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.602714
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.596419
Train Epoch: 71 - Loss: 0.6292, Accuracy: 94.78%

Test set: Average loss: 0.5183, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.672451
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.658798
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.671061
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.625700
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.604530
Train Epoch: 72 - Loss: 0.6298, Accuracy: 94.72%

Test set: Average loss: 0.5183, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.720172
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.599713
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.620391
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.604203
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.637064
Train Epoch: 73 - Loss: 0.6303, Accuracy: 94.76%

Test set: Average loss: 0.5178, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.626054
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.594597
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.719796
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.696167
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.654113
Train Epoch: 74 - Loss: 0.6285, Accuracy: 94.86%

Test set: Average loss: 0.5180, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.716493
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.631354
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.620794
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.638325
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.596698
Train Epoch: 75 - Loss: 0.6274, Accuracy: 94.88%

Test set: Average loss: 0.5179, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.580955
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.625333
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.648971
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.617367
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.625811
Train Epoch: 76 - Loss: 0.6398, Accuracy: 94.38%

Test set: Average loss: 0.5215, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.683348
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.614935
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.610075
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.639559
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.661203
Train Epoch: 77 - Loss: 0.6388, Accuracy: 94.35%

Test set: Average loss: 0.5194, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.627163
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.639796
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.664195
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.584361
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.623911
Train Epoch: 78 - Loss: 0.6367, Accuracy: 94.45%

Test set: Average loss: 0.5198, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.632536
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.639445
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.632236
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.620575
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.677010
Train Epoch: 79 - Loss: 0.6350, Accuracy: 94.60%

Test set: Average loss: 0.5184, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.617765
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.739148
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.663960
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.666778
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.617417
Train Epoch: 80 - Loss: 0.6365, Accuracy: 94.45%

Test set: Average loss: 0.5190, Accuracy: 9956/10000 (99.56%)

New best model saved with accuracy: 99.56%
Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.586944
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.630159
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.641200
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.611015
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.641100
Train Epoch: 81 - Loss: 0.6360, Accuracy: 94.50%

Test set: Average loss: 0.5186, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.621864
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.583268
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.658996
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.662358
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.621478
Train Epoch: 82 - Loss: 0.6350, Accuracy: 94.52%

Test set: Average loss: 0.5184, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.652327
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.565097
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.614567
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.676543
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.600912
Train Epoch: 83 - Loss: 0.6320, Accuracy: 94.70%

Test set: Average loss: 0.5189, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.597556
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.620013
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.663757
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.544928
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.727956
Train Epoch: 84 - Loss: 0.6356, Accuracy: 94.50%

Test set: Average loss: 0.5184, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.651453
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.644782
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.656101
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.623634
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.623184
Train Epoch: 85 - Loss: 0.6331, Accuracy: 94.69%

Test set: Average loss: 0.5178, Accuracy: 9960/10000 (99.60%)

New best model saved with accuracy: 99.60%
Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.641129
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.660676
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.747234
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.656206
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.600149
Train Epoch: 86 - Loss: 0.6292, Accuracy: 94.78%

Test set: Average loss: 0.5174, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.631796
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.636080
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.622211
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.598808
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.572895
Train Epoch: 87 - Loss: 0.6301, Accuracy: 94.74%

Test set: Average loss: 0.5178, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.622267
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.578880
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.649758
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.626158
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.695522
Train Epoch: 88 - Loss: 0.6292, Accuracy: 94.69%

Test set: Average loss: 0.5170, Accuracy: 9961/10000 (99.61%)

New best model saved with accuracy: 99.61%
Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.590147
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.603419
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.628620
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.609088
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.585263
Train Epoch: 89 - Loss: 0.6282, Accuracy: 94.80%

Test set: Average loss: 0.5173, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.622400
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.632257
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.670801
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.606101
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.613821
Train Epoch: 90 - Loss: 0.6266, Accuracy: 94.95%

Test set: Average loss: 0.5175, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.622468
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.597333
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.566455
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.658728
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.599874
Train Epoch: 91 - Loss: 0.6254, Accuracy: 94.86%

Test set: Average loss: 0.5165, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.652129
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.653223
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.620816
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.692166
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.673183
Train Epoch: 92 - Loss: 0.6252, Accuracy: 94.89%

Test set: Average loss: 0.5168, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.651208
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.667591
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.702513
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.606426
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.611547
Train Epoch: 93 - Loss: 0.6258, Accuracy: 94.91%

Test set: Average loss: 0.5163, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.656091
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.602246
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.648358
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.701772
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.571818
Train Epoch: 94 - Loss: 0.6235, Accuracy: 95.01%

Test set: Average loss: 0.5159, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.625640
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.637182
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.665826
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.618768
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.667880
Train Epoch: 95 - Loss: 0.6237, Accuracy: 94.97%

Test set: Average loss: 0.5160, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.622313
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.594636
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.620892
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.606676
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.586813
Train Epoch: 96 - Loss: 0.6224, Accuracy: 95.10%

Test set: Average loss: 0.5159, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.637852
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.590988
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.579954
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.562333
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.589721
Train Epoch: 97 - Loss: 0.6232, Accuracy: 94.93%

Test set: Average loss: 0.5160, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.658153
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.606296
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.597221
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.674108
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.576750
Train Epoch: 98 - Loss: 0.6213, Accuracy: 95.08%

Test set: Average loss: 0.5160, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.632162
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.602609
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.615381
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.559127
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.593275
Train Epoch: 99 - Loss: 0.6207, Accuracy: 95.02%

Test set: Average loss: 0.5159, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.572036
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.605982
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.627017
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.658683
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.612982
Train Epoch: 100 - Loss: 0.6205, Accuracy: 95.13%

Test set: Average loss: 0.5159, Accuracy: 9957/10000 (99.57%)


Training completed. Best Accuracy: 99.61%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.190193
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.702590
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.271013
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.096252
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.963252
Train Epoch: 1 - Loss: 1.3809, Accuracy: 69.17%

Test set: Average loss: 0.6794, Accuracy: 9638/10000 (96.38%)

New best model saved with accuracy: 96.38%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.919565
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.917312
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.844859
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.789110
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.801146
Train Epoch: 2 - Loss: 0.8829, Accuracy: 85.61%

Test set: Average loss: 0.6023, Accuracy: 9821/10000 (98.21%)

New best model saved with accuracy: 98.21%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.840744
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.826099
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.843828
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.805446
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.734516
Train Epoch: 3 - Loss: 0.8112, Accuracy: 88.39%

Test set: Average loss: 0.5978, Accuracy: 9770/10000 (97.70%)

Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.757301
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.753265
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.758166
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.751127
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.687554
Train Epoch: 4 - Loss: 0.7721, Accuracy: 89.82%

Test set: Average loss: 0.5837, Accuracy: 9833/10000 (98.33%)

New best model saved with accuracy: 98.33%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.839670
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.740814
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.699391
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.779402
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.778575
Train Epoch: 5 - Loss: 0.7524, Accuracy: 90.51%

Test set: Average loss: 0.5607, Accuracy: 9874/10000 (98.74%)

New best model saved with accuracy: 98.74%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.725425
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.704970
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.703286
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.777146
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.685241
Train Epoch: 6 - Loss: 0.7320, Accuracy: 91.17%

Test set: Average loss: 0.5568, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.673581
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.830252
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.706061
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.765633
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.663983
Train Epoch: 7 - Loss: 0.7253, Accuracy: 91.38%

Test set: Average loss: 0.5458, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.663185
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.714964
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.714601
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.756954
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.663952
Train Epoch: 8 - Loss: 0.7149, Accuracy: 91.90%

Test set: Average loss: 0.5465, Accuracy: 9903/10000 (99.03%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.650148
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.672564
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.747529
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.724712
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.674830
Train Epoch: 9 - Loss: 0.7073, Accuracy: 92.04%

Test set: Average loss: 0.5454, Accuracy: 9907/10000 (99.07%)

Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.726783
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.726349
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.766555
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.716847
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.653977
Train Epoch: 10 - Loss: 0.7005, Accuracy: 92.33%

Test set: Average loss: 0.5384, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.770996
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.650870
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.656165
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.723022
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.697054
Train Epoch: 11 - Loss: 0.6958, Accuracy: 92.49%

Test set: Average loss: 0.5371, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.636250
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.712611
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.694385
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.704241
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.615472
Train Epoch: 12 - Loss: 0.6911, Accuracy: 92.65%

Test set: Average loss: 0.5361, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.649126
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.645649
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.628430
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.640632
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.737172
Train Epoch: 13 - Loss: 0.6860, Accuracy: 92.82%

Test set: Average loss: 0.5351, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.725368
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.675325
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.662065
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.632780
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.700985
Train Epoch: 14 - Loss: 0.6840, Accuracy: 92.89%

Test set: Average loss: 0.5355, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.707551
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.631727
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.758649
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.660053
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.648510
Train Epoch: 15 - Loss: 0.6818, Accuracy: 92.95%

Test set: Average loss: 0.5342, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.600368
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.621690
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.611720
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.759589
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.667218
Train Epoch: 16 - Loss: 0.6792, Accuracy: 93.03%

Test set: Average loss: 0.5330, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.723058
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.666697
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.656369
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.664393
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.609383
Train Epoch: 17 - Loss: 0.6769, Accuracy: 93.24%

Test set: Average loss: 0.5309, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.622320
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.643785
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.617673
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.783148
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.624973
Train Epoch: 18 - Loss: 0.6753, Accuracy: 93.22%

Test set: Average loss: 0.5321, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.684201
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.701250
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.697314
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.642135
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.702488
Train Epoch: 19 - Loss: 0.6716, Accuracy: 93.29%

Test set: Average loss: 0.5312, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.594314
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.700166
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.783692
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.700581
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.695387
Train Epoch: 20 - Loss: 0.6697, Accuracy: 93.48%

Test set: Average loss: 0.5303, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.730119
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.639507
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.764755
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.679669
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.659847
Train Epoch: 21 - Loss: 0.6693, Accuracy: 93.45%

Test set: Average loss: 0.5294, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.710319
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.677940
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.648710
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.682946
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.670577
Train Epoch: 22 - Loss: 0.6710, Accuracy: 93.51%

Test set: Average loss: 0.5291, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.647948
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.662140
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.667479
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.660675
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.684889
Train Epoch: 23 - Loss: 0.6719, Accuracy: 93.33%

Test set: Average loss: 0.5300, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.654951
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.711301
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.656352
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.668467
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.627884
Train Epoch: 24 - Loss: 0.6692, Accuracy: 93.48%

Test set: Average loss: 0.5294, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.665485
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.664708
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.730719
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.648662
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.687984
Train Epoch: 25 - Loss: 0.6698, Accuracy: 93.36%

Test set: Average loss: 0.5298, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.658079
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.712646
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.656063
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.684611
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.636683
Train Epoch: 26 - Loss: 0.6778, Accuracy: 93.07%

Test set: Average loss: 0.5312, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.656114
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.653874
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.617686
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.695892
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.645305
Train Epoch: 27 - Loss: 0.6784, Accuracy: 93.12%

Test set: Average loss: 0.5313, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.601749
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.698300
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.629689
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.648348
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.684341
Train Epoch: 28 - Loss: 0.6741, Accuracy: 93.25%

Test set: Average loss: 0.5317, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.668292
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.607904
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.631140
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.668294
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.661661
Train Epoch: 29 - Loss: 0.6732, Accuracy: 93.23%

Test set: Average loss: 0.5316, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.634105
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.702909
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.711697
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.647109
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.704772
Train Epoch: 30 - Loss: 0.6702, Accuracy: 93.35%

Test set: Average loss: 0.5285, Accuracy: 9945/10000 (99.45%)

New best model saved with accuracy: 99.45%
Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.711009
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.663870
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.651488
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.656035
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.596874
Train Epoch: 31 - Loss: 0.6661, Accuracy: 93.48%

Test set: Average loss: 0.5298, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.613768
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.655045
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.593936
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.729621
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.675855
Train Epoch: 32 - Loss: 0.6659, Accuracy: 93.46%

Test set: Average loss: 0.5280, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.638038
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.642992
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.732856
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.671230
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.628893
Train Epoch: 33 - Loss: 0.6618, Accuracy: 93.67%

Test set: Average loss: 0.5275, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.667767
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.660299
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.649346
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.656454
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.663721
Train Epoch: 34 - Loss: 0.6601, Accuracy: 93.63%

Test set: Average loss: 0.5288, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.677478
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.662401
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.612241
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.699273
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.697657
Train Epoch: 35 - Loss: 0.6586, Accuracy: 93.75%

Test set: Average loss: 0.5252, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.610990
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.730054
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.693481
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.660922
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.685227
Train Epoch: 36 - Loss: 0.6576, Accuracy: 93.83%

Test set: Average loss: 0.5252, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.636812
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.668842
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.673145
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.664463
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.603180
Train Epoch: 37 - Loss: 0.6567, Accuracy: 93.83%

Test set: Average loss: 0.5248, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.585641
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.577544
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.638781
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.648856
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.733051
Train Epoch: 38 - Loss: 0.6535, Accuracy: 93.89%

Test set: Average loss: 0.5238, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.657008
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.631284
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.598798
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.601650
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.710181
Train Epoch: 39 - Loss: 0.6520, Accuracy: 94.03%

Test set: Average loss: 0.5241, Accuracy: 9954/10000 (99.54%)

New best model saved with accuracy: 99.54%
Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.688190
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.702830
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.717678
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.681030
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.699669
Train Epoch: 40 - Loss: 0.6523, Accuracy: 93.98%

Test set: Average loss: 0.5224, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.649016
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.616345
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.701871
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.748457
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.663101
Train Epoch: 41 - Loss: 0.6486, Accuracy: 94.19%

Test set: Average loss: 0.5223, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.632668
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.597285
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.667461
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.662691
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.684912
Train Epoch: 42 - Loss: 0.6478, Accuracy: 94.22%

Test set: Average loss: 0.5237, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.582817
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.658624
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.567249
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.662676
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.695218
Train Epoch: 43 - Loss: 0.6483, Accuracy: 94.12%

Test set: Average loss: 0.5222, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.599554
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.620751
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.643611
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.604020
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.691707
Train Epoch: 44 - Loss: 0.6422, Accuracy: 94.31%

Test set: Average loss: 0.5219, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.585174
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.599050
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.657552
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.655924
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.630036
Train Epoch: 45 - Loss: 0.6442, Accuracy: 94.30%

Test set: Average loss: 0.5219, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.650505
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.620401
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.731710
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.650951
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.622490
Train Epoch: 46 - Loss: 0.6446, Accuracy: 94.21%

Test set: Average loss: 0.5220, Accuracy: 9955/10000 (99.55%)

New best model saved with accuracy: 99.55%
Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.648356
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.591862
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.647271
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.633263
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.656332
Train Epoch: 47 - Loss: 0.6434, Accuracy: 94.37%

Test set: Average loss: 0.5217, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.606061
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.615152
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.697628
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.688138
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.635718
Train Epoch: 48 - Loss: 0.6451, Accuracy: 94.23%

Test set: Average loss: 0.5214, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.669535
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.644135
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.706316
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.645700
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.635293
Train Epoch: 49 - Loss: 0.6416, Accuracy: 94.39%

Test set: Average loss: 0.5215, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.602985
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.671031
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.609275
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.652858
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.632740
Train Epoch: 50 - Loss: 0.6428, Accuracy: 94.31%

Test set: Average loss: 0.5215, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.613295
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.623913
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.689053
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.702128
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.668188
Train Epoch: 51 - Loss: 0.6546, Accuracy: 93.82%

Test set: Average loss: 0.5237, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.648341
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.650648
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.636097
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.642893
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.573746
Train Epoch: 52 - Loss: 0.6548, Accuracy: 93.90%

Test set: Average loss: 0.5234, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.669604
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.635168
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.608205
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.628568
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.635329
Train Epoch: 53 - Loss: 0.6522, Accuracy: 93.87%

Test set: Average loss: 0.5222, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.655164
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.615807
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.697919
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.711897
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.608695
Train Epoch: 54 - Loss: 0.6517, Accuracy: 93.99%

Test set: Average loss: 0.5234, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.700332
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.613810
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.622948
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.645497
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.634834
Train Epoch: 55 - Loss: 0.6533, Accuracy: 93.75%

Test set: Average loss: 0.5234, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.622322
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.608906
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.595115
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.617706
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.646976
Train Epoch: 56 - Loss: 0.6472, Accuracy: 94.09%

Test set: Average loss: 0.5221, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.576252
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.639092
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.646030
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.672580
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.628172
Train Epoch: 57 - Loss: 0.6451, Accuracy: 94.19%

Test set: Average loss: 0.5231, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.665346
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.616026
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.638012
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.702056
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.613559
Train Epoch: 58 - Loss: 0.6453, Accuracy: 94.17%

Test set: Average loss: 0.5211, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.654384
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.647928
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.718162
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.670776
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.699648
Train Epoch: 59 - Loss: 0.6456, Accuracy: 94.22%

Test set: Average loss: 0.5209, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.618631
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.640485
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.649966
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.633969
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.602474
Train Epoch: 60 - Loss: 0.6432, Accuracy: 94.26%

Test set: Average loss: 0.5203, Accuracy: 9958/10000 (99.58%)

New best model saved with accuracy: 99.58%
Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.621966
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.613412
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.685632
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.629814
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.620517
Train Epoch: 61 - Loss: 0.6420, Accuracy: 94.38%

Test set: Average loss: 0.5206, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.630387
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.628458
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.619894
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.725959
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.624219
Train Epoch: 62 - Loss: 0.6420, Accuracy: 94.26%

Test set: Average loss: 0.5204, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.613505
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.589510
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.615174
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.619814
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.637167
Train Epoch: 63 - Loss: 0.6390, Accuracy: 94.39%

Test set: Average loss: 0.5201, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.638297
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.616412
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.744138
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.665524
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.662784
Train Epoch: 64 - Loss: 0.6406, Accuracy: 94.31%

Test set: Average loss: 0.5195, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.644196
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.634260
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.666986
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.657762
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.728237
Train Epoch: 65 - Loss: 0.6379, Accuracy: 94.53%

Test set: Average loss: 0.5202, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.717104
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.618156
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.597147
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.579165
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.623597
Train Epoch: 66 - Loss: 0.6361, Accuracy: 94.53%

Test set: Average loss: 0.5194, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.647317
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.639715
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.604030
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.595355
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.620095
Train Epoch: 67 - Loss: 0.6380, Accuracy: 94.48%

Test set: Average loss: 0.5192, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.600836
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.690018
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.591125
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.703045
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.708116
Train Epoch: 68 - Loss: 0.6350, Accuracy: 94.59%

Test set: Average loss: 0.5187, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.620703
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.620802
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.613615
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.669779
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.610357
Train Epoch: 69 - Loss: 0.6341, Accuracy: 94.67%

Test set: Average loss: 0.5188, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.597468
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.609145
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.625425
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.645263
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.622442
Train Epoch: 70 - Loss: 0.6307, Accuracy: 94.80%

Test set: Average loss: 0.5184, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.610083
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.609507
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.572435
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.682057
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.656607
Train Epoch: 71 - Loss: 0.6314, Accuracy: 94.75%

Test set: Average loss: 0.5182, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.637532
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.626610
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.650729
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.609335
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.576932
Train Epoch: 72 - Loss: 0.6331, Accuracy: 94.67%

Test set: Average loss: 0.5187, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.588342
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.586777
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.630719
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.686598
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.644388
Train Epoch: 73 - Loss: 0.6316, Accuracy: 94.86%

Test set: Average loss: 0.5186, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.603371
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.625495
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.662663
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.671975
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.638865
Train Epoch: 74 - Loss: 0.6325, Accuracy: 94.66%

Test set: Average loss: 0.5186, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.601910
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.637487
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.621042
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.634061
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.617238
Train Epoch: 75 - Loss: 0.6316, Accuracy: 94.73%

Test set: Average loss: 0.5185, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.598202
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.699687
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.677780
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.647389
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.612320
Train Epoch: 76 - Loss: 0.6394, Accuracy: 94.44%

Test set: Average loss: 0.5207, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.647982
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.595668
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.594347
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.668450
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.642012
Train Epoch: 77 - Loss: 0.6415, Accuracy: 94.25%

Test set: Average loss: 0.5197, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.582162
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.627411
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.638713
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.668482
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.613716
Train Epoch: 78 - Loss: 0.6414, Accuracy: 94.31%

Test set: Average loss: 0.5196, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.637307
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.594746
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.706928
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.623507
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.631285
Train Epoch: 79 - Loss: 0.6427, Accuracy: 94.24%

Test set: Average loss: 0.5201, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.680193
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.633405
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.604299
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.595854
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.668710
Train Epoch: 80 - Loss: 0.6401, Accuracy: 94.31%

Test set: Average loss: 0.5212, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.645237
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.649590
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.585270
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.691670
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.630962
Train Epoch: 81 - Loss: 0.6394, Accuracy: 94.39%

Test set: Average loss: 0.5193, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.663357
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.636505
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.660120
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.677293
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.684687
Train Epoch: 82 - Loss: 0.6400, Accuracy: 94.21%

Test set: Average loss: 0.5188, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.622080
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.711642
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.600481
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.624969
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.588088
Train Epoch: 83 - Loss: 0.6357, Accuracy: 94.50%

Test set: Average loss: 0.5196, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.597804
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.646997
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.615602
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.621992
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.627707
Train Epoch: 84 - Loss: 0.6358, Accuracy: 94.54%

Test set: Average loss: 0.5190, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.634358
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.659752
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.628677
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.624724
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.642245
Train Epoch: 85 - Loss: 0.6338, Accuracy: 94.56%

Test set: Average loss: 0.5201, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.642570
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.659589
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.609861
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.667340
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.602062
Train Epoch: 86 - Loss: 0.6326, Accuracy: 94.72%

Test set: Average loss: 0.5188, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.697242
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.598942
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.657631
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.638310
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.662488
Train Epoch: 87 - Loss: 0.6324, Accuracy: 94.68%

Test set: Average loss: 0.5177, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.576275
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.633557
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.667964
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.629462
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.623905
Train Epoch: 88 - Loss: 0.6316, Accuracy: 94.60%

Test set: Average loss: 0.5178, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.684694
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.586609
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.608473
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.662179
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.671799
Train Epoch: 89 - Loss: 0.6296, Accuracy: 94.70%

Test set: Average loss: 0.5180, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.660583
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.679244
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.564107
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.641680
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.694387
Train Epoch: 90 - Loss: 0.6304, Accuracy: 94.73%

Test set: Average loss: 0.5175, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.647452
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.625458
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.608549
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.592994
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.614431
Train Epoch: 91 - Loss: 0.6292, Accuracy: 94.76%

Test set: Average loss: 0.5176, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.682011
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.610775
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.680751
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.603740
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.615482
Train Epoch: 92 - Loss: 0.6291, Accuracy: 94.77%

Test set: Average loss: 0.5175, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.595144
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.623465
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.654282
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.587845
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.645467
Train Epoch: 93 - Loss: 0.6292, Accuracy: 94.83%

Test set: Average loss: 0.5167, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.641389
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.599194
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.641383
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.594381
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.665581
Train Epoch: 94 - Loss: 0.6267, Accuracy: 94.90%

Test set: Average loss: 0.5175, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.689556
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.641594
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.598168
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.647305
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.617287
Train Epoch: 95 - Loss: 0.6259, Accuracy: 94.92%

Test set: Average loss: 0.5169, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.623079
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.620507
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.601252
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.630595
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.626601
Train Epoch: 96 - Loss: 0.6246, Accuracy: 95.00%

Test set: Average loss: 0.5169, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.682242
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.662253
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.589525
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.624975
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.627699
Train Epoch: 97 - Loss: 0.6270, Accuracy: 94.92%

Test set: Average loss: 0.5167, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.650027
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.668394
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.638326
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.608955
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.614075
Train Epoch: 98 - Loss: 0.6272, Accuracy: 94.83%

Test set: Average loss: 0.5167, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.630926
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.595915
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.620121
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.627033
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.624577
Train Epoch: 99 - Loss: 0.6267, Accuracy: 94.90%

Test set: Average loss: 0.5166, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.592434
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.600267
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.577064
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.646138
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.621968
Train Epoch: 100 - Loss: 0.6249, Accuracy: 94.92%

Test set: Average loss: 0.5167, Accuracy: 9955/10000 (99.55%)


Training completed. Best Accuracy: 99.58%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.605229
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.349372
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.134986
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.091765
Train Epoch: 1 [51200/60000 (85%)]	Loss: 1.080423
Train Epoch: 1 - Loss: 1.3348, Accuracy: 69.79%

Test set: Average loss: 0.6587, Accuracy: 9636/10000 (96.36%)

New best model saved with accuracy: 96.36%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.969731
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.972541
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.847804
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.856013
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.823387
Train Epoch: 2 - Loss: 0.8785, Accuracy: 85.66%

Test set: Average loss: 0.6132, Accuracy: 9745/10000 (97.45%)

New best model saved with accuracy: 97.45%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.893497
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.874071
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.792752
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.775207
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.774223
Train Epoch: 3 - Loss: 0.8083, Accuracy: 88.33%

Test set: Average loss: 0.5789, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.760400
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.789373
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.785503
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.693231
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.798432
Train Epoch: 4 - Loss: 0.7713, Accuracy: 89.60%

Test set: Average loss: 0.5653, Accuracy: 9848/10000 (98.48%)

New best model saved with accuracy: 98.48%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.685376
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.724602
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.816398
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.730105
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.722458
Train Epoch: 5 - Loss: 0.7523, Accuracy: 90.22%

Test set: Average loss: 0.5572, Accuracy: 9868/10000 (98.68%)

New best model saved with accuracy: 98.68%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.685498
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.698471
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.695764
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.779706
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.754727
Train Epoch: 6 - Loss: 0.7369, Accuracy: 91.01%

Test set: Average loss: 0.5519, Accuracy: 9877/10000 (98.77%)

New best model saved with accuracy: 98.77%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.707878
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.737153
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.700985
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.681082
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.695156
Train Epoch: 7 - Loss: 0.7249, Accuracy: 91.34%

Test set: Average loss: 0.5531, Accuracy: 9874/10000 (98.74%)

Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.680513
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.700139
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.699582
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.658272
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.700723
Train Epoch: 8 - Loss: 0.7144, Accuracy: 91.75%

Test set: Average loss: 0.5494, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.707939
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.871398
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.668139
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.722732
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.694838
Train Epoch: 9 - Loss: 0.7076, Accuracy: 92.17%

Test set: Average loss: 0.5449, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.744675
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.675535
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.737650
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.735259
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.723699
Train Epoch: 10 - Loss: 0.6982, Accuracy: 92.35%

Test set: Average loss: 0.5405, Accuracy: 9905/10000 (99.05%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.798944
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.610033
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.657964
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.713544
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.645362
Train Epoch: 11 - Loss: 0.6952, Accuracy: 92.42%

Test set: Average loss: 0.5380, Accuracy: 9910/10000 (99.10%)

New best model saved with accuracy: 99.10%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.698017
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.698268
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.725662
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.672704
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.657922
Train Epoch: 12 - Loss: 0.6903, Accuracy: 92.60%

Test set: Average loss: 0.5362, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.751266
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.647574
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.720163
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.694891
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.683307
Train Epoch: 13 - Loss: 0.6872, Accuracy: 92.72%

Test set: Average loss: 0.5392, Accuracy: 9910/10000 (99.10%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.663025
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.725936
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.680027
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.726944
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.689827
Train Epoch: 14 - Loss: 0.6837, Accuracy: 92.93%

Test set: Average loss: 0.5338, Accuracy: 9918/10000 (99.18%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.682166
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.678124
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.719939
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.640221
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.680362
Train Epoch: 15 - Loss: 0.6782, Accuracy: 93.15%

Test set: Average loss: 0.5343, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.691993
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.729546
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.702690
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.652527
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.663556
Train Epoch: 16 - Loss: 0.6775, Accuracy: 93.08%

Test set: Average loss: 0.5350, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.605365
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.670265
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.672803
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.678650
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.662211
Train Epoch: 17 - Loss: 0.6739, Accuracy: 93.16%

Test set: Average loss: 0.5324, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.708957
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.648036
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.649389
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.679935
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.662866
Train Epoch: 18 - Loss: 0.6734, Accuracy: 93.29%

Test set: Average loss: 0.5315, Accuracy: 9926/10000 (99.26%)

New best model saved with accuracy: 99.26%
Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.698820
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.647333
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.639249
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.713114
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.646887
Train Epoch: 19 - Loss: 0.6734, Accuracy: 93.22%

Test set: Average loss: 0.5316, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.711472
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.627479
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.652744
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.688519
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.620865
Train Epoch: 20 - Loss: 0.6716, Accuracy: 93.28%

Test set: Average loss: 0.5310, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.661958
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.746616
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.615160
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.672390
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.652938
Train Epoch: 21 - Loss: 0.6697, Accuracy: 93.33%

Test set: Average loss: 0.5307, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.613211
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.746115
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.737051
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.688311
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.681435
Train Epoch: 22 - Loss: 0.6690, Accuracy: 93.38%

Test set: Average loss: 0.5306, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.714670
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.675501
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.682147
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.651490
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.688136
Train Epoch: 23 - Loss: 0.6673, Accuracy: 93.47%

Test set: Average loss: 0.5302, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.760842
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.655706
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.664404
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.625846
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.655310
Train Epoch: 24 - Loss: 0.6658, Accuracy: 93.58%

Test set: Average loss: 0.5305, Accuracy: 9924/10000 (99.24%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.659661
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.702018
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.682911
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.676201
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.663612
Train Epoch: 25 - Loss: 0.6673, Accuracy: 93.43%

Test set: Average loss: 0.5302, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.669429
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.622051
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.667599
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.609477
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.649764
Train Epoch: 26 - Loss: 0.6796, Accuracy: 92.92%

Test set: Average loss: 0.5353, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.624153
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.641863
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.622811
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.645787
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.724347
Train Epoch: 27 - Loss: 0.6792, Accuracy: 92.97%

Test set: Average loss: 0.5328, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.714340
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.714832
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.759878
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.712204
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.700296
Train Epoch: 28 - Loss: 0.6748, Accuracy: 93.12%

Test set: Average loss: 0.5321, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.724457
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.664629
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.695805
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.738633
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.716098
Train Epoch: 29 - Loss: 0.6729, Accuracy: 93.16%

Test set: Average loss: 0.5308, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.630937
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.714514
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.700600
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.674387
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.668684
Train Epoch: 30 - Loss: 0.6708, Accuracy: 93.30%

Test set: Average loss: 0.5293, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.642564
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.766356
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.692626
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.651436
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.672728
Train Epoch: 31 - Loss: 0.6630, Accuracy: 93.65%

Test set: Average loss: 0.5298, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.669515
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.641514
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.622283
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.613597
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.814954
Train Epoch: 32 - Loss: 0.6609, Accuracy: 93.65%

Test set: Average loss: 0.5316, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.631382
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.674983
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.657451
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.672370
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.672831
Train Epoch: 33 - Loss: 0.6611, Accuracy: 93.55%

Test set: Average loss: 0.5263, Accuracy: 9935/10000 (99.35%)

New best model saved with accuracy: 99.35%
Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.693865
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.639344
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.746147
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.699057
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.666933
Train Epoch: 34 - Loss: 0.6585, Accuracy: 93.82%

Test set: Average loss: 0.5275, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.632368
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.688405
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.654005
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.680966
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.641537
Train Epoch: 35 - Loss: 0.6562, Accuracy: 93.94%

Test set: Average loss: 0.5255, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.597509
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.689965
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.661033
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.644892
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.645015
Train Epoch: 36 - Loss: 0.6549, Accuracy: 93.89%

Test set: Average loss: 0.5252, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.659485
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.596457
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.709308
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.625998
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.775624
Train Epoch: 37 - Loss: 0.6536, Accuracy: 93.95%

Test set: Average loss: 0.5247, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.733982
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.696612
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.701088
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.614230
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.687953
Train Epoch: 38 - Loss: 0.6503, Accuracy: 94.05%

Test set: Average loss: 0.5258, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.700652
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.676964
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.668603
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.619252
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.691115
Train Epoch: 39 - Loss: 0.6500, Accuracy: 94.04%

Test set: Average loss: 0.5241, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.634992
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.626134
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.634858
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.701032
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.614899
Train Epoch: 40 - Loss: 0.6486, Accuracy: 94.21%

Test set: Average loss: 0.5237, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.627005
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.638862
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.602826
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.686225
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.706587
Train Epoch: 41 - Loss: 0.6491, Accuracy: 94.11%

Test set: Average loss: 0.5241, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.644763
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.611938
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.631427
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.608696
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.668001
Train Epoch: 42 - Loss: 0.6455, Accuracy: 94.23%

Test set: Average loss: 0.5233, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.656481
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.604447
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.621554
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.612188
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.669124
Train Epoch: 43 - Loss: 0.6421, Accuracy: 94.42%

Test set: Average loss: 0.5230, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.614897
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.639165
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.609125
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.579525
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.656129
Train Epoch: 44 - Loss: 0.6430, Accuracy: 94.37%

Test set: Average loss: 0.5229, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.687806
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.733458
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.641786
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.628738
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.697591
Train Epoch: 45 - Loss: 0.6455, Accuracy: 94.21%

Test set: Average loss: 0.5230, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.655164
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.684754
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.607911
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.647952
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.692861
Train Epoch: 46 - Loss: 0.6405, Accuracy: 94.50%

Test set: Average loss: 0.5222, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.651634
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.615595
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.645926
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.665875
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.648858
Train Epoch: 47 - Loss: 0.6415, Accuracy: 94.31%

Test set: Average loss: 0.5221, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.657018
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.622316
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.670295
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.660860
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.708967
Train Epoch: 48 - Loss: 0.6399, Accuracy: 94.44%

Test set: Average loss: 0.5221, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.645547
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.614940
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.661247
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.632718
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.610341
Train Epoch: 49 - Loss: 0.6418, Accuracy: 94.34%

Test set: Average loss: 0.5222, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.665170
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.638865
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.603762
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.640345
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.643951
Train Epoch: 50 - Loss: 0.6398, Accuracy: 94.47%

Test set: Average loss: 0.5220, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.652663
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.662722
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.614704
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.659910
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.593289
Train Epoch: 51 - Loss: 0.6544, Accuracy: 93.83%

Test set: Average loss: 0.5279, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.634251
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.598738
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.693221
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.708006
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.671625
Train Epoch: 52 - Loss: 0.6506, Accuracy: 93.98%

Test set: Average loss: 0.5276, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.628739
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.685493
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.638255
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.622488
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.593866
Train Epoch: 53 - Loss: 0.6508, Accuracy: 93.95%

Test set: Average loss: 0.5236, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.656783
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.595750
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.735772
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.699175
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.727433
Train Epoch: 54 - Loss: 0.6497, Accuracy: 94.01%

Test set: Average loss: 0.5227, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.702205
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.658455
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.627367
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.647942
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.591431
Train Epoch: 55 - Loss: 0.6508, Accuracy: 93.98%

Test set: Average loss: 0.5227, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.739869
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.659350
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.660163
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.653026
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.691148
Train Epoch: 56 - Loss: 0.6483, Accuracy: 94.00%

Test set: Average loss: 0.5221, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.646910
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.646773
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.636094
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.659123
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.607380
Train Epoch: 57 - Loss: 0.6455, Accuracy: 94.14%

Test set: Average loss: 0.5220, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.669390
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.607100
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.703558
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.617697
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.688412
Train Epoch: 58 - Loss: 0.6452, Accuracy: 94.14%

Test set: Average loss: 0.5215, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.599847
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.645244
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.647110
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.601846
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.670976
Train Epoch: 59 - Loss: 0.6410, Accuracy: 94.35%

Test set: Average loss: 0.5216, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.682052
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.678493
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.591568
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.648569
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.656629
Train Epoch: 60 - Loss: 0.6398, Accuracy: 94.38%

Test set: Average loss: 0.5223, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.665500
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.631543
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.638665
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.643451
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.681529
Train Epoch: 61 - Loss: 0.6403, Accuracy: 94.38%

Test set: Average loss: 0.5210, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.678234
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.637100
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.635768
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.616182
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.654048
Train Epoch: 62 - Loss: 0.6426, Accuracy: 94.30%

Test set: Average loss: 0.5209, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.663685
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.564718
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.690879
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.652446
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.603391
Train Epoch: 63 - Loss: 0.6380, Accuracy: 94.36%

Test set: Average loss: 0.5204, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.565023
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.644095
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.714474
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.642923
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.642426
Train Epoch: 64 - Loss: 0.6369, Accuracy: 94.42%

Test set: Average loss: 0.5202, Accuracy: 9954/10000 (99.54%)

New best model saved with accuracy: 99.54%
Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.657486
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.702536
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.608091
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.579105
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.648785
Train Epoch: 65 - Loss: 0.6384, Accuracy: 94.44%

Test set: Average loss: 0.5196, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.626587
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.697732
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.641552
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.595106
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.690788
Train Epoch: 66 - Loss: 0.6358, Accuracy: 94.60%

Test set: Average loss: 0.5201, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.618801
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.573595
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.628686
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.648974
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.570713
Train Epoch: 67 - Loss: 0.6369, Accuracy: 94.36%

Test set: Average loss: 0.5196, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.636339
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.651129
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.582561
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.679264
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.642738
Train Epoch: 68 - Loss: 0.6326, Accuracy: 94.67%

Test set: Average loss: 0.5192, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.625040
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.638785
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.635299
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.623699
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.696270
Train Epoch: 69 - Loss: 0.6317, Accuracy: 94.72%

Test set: Average loss: 0.5194, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.626923
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.680317
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.615449
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.616957
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.640151
Train Epoch: 70 - Loss: 0.6307, Accuracy: 94.72%

Test set: Average loss: 0.5193, Accuracy: 9955/10000 (99.55%)

New best model saved with accuracy: 99.55%
Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.589161
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.659517
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.568121
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.707391
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.665323
Train Epoch: 71 - Loss: 0.6318, Accuracy: 94.69%

Test set: Average loss: 0.5192, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.588732
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.620201
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.619883
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.612706
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.626280
Train Epoch: 72 - Loss: 0.6311, Accuracy: 94.70%

Test set: Average loss: 0.5194, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.657288
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.666293
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.590582
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.659304
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.640818
Train Epoch: 73 - Loss: 0.6313, Accuracy: 94.59%

Test set: Average loss: 0.5195, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.617823
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.658877
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.656597
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.582520
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.701490
Train Epoch: 74 - Loss: 0.6288, Accuracy: 94.80%

Test set: Average loss: 0.5190, Accuracy: 9956/10000 (99.56%)

New best model saved with accuracy: 99.56%
Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.641906
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.681608
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.682237
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.607425
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.609341
Train Epoch: 75 - Loss: 0.6306, Accuracy: 94.66%

Test set: Average loss: 0.5188, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.652107
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.661124
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.615317
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.612838
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.633649
Train Epoch: 76 - Loss: 0.6415, Accuracy: 94.27%

Test set: Average loss: 0.5217, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.671914
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.707689
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.653506
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.674140
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.585856
Train Epoch: 77 - Loss: 0.6408, Accuracy: 94.31%

Test set: Average loss: 0.5198, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.622020
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.606626
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.665656
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.650989
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.658809
Train Epoch: 78 - Loss: 0.6382, Accuracy: 94.48%

Test set: Average loss: 0.5218, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.630846
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.645268
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.630495
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.638575
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.636315
Train Epoch: 79 - Loss: 0.6409, Accuracy: 94.31%

Test set: Average loss: 0.5207, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.700236
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.602541
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.628736
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.685756
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.671430
Train Epoch: 80 - Loss: 0.6385, Accuracy: 94.42%

Test set: Average loss: 0.5207, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.628118
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.589437
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.688658
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.568001
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.615171
Train Epoch: 81 - Loss: 0.6358, Accuracy: 94.53%

Test set: Average loss: 0.5199, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.614913
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.628955
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.629104
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.688351
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.594636
Train Epoch: 82 - Loss: 0.6372, Accuracy: 94.49%

Test set: Average loss: 0.5192, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.620003
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.603419
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.677146
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.649138
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.750054
Train Epoch: 83 - Loss: 0.6375, Accuracy: 94.42%

Test set: Average loss: 0.5197, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.610436
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.640873
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.612861
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.611307
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.640276
Train Epoch: 84 - Loss: 0.6352, Accuracy: 94.54%

Test set: Average loss: 0.5190, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.651319
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.712277
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.631712
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.610851
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.640059
Train Epoch: 85 - Loss: 0.6339, Accuracy: 94.58%

Test set: Average loss: 0.5189, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.624046
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.643758
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.602548
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.639831
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.617050
Train Epoch: 86 - Loss: 0.6316, Accuracy: 94.63%

Test set: Average loss: 0.5196, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.674637
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.670777
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.606598
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.604653
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.675277
Train Epoch: 87 - Loss: 0.6347, Accuracy: 94.44%

Test set: Average loss: 0.5185, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.657837
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.667062
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.681794
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.578568
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.650446
Train Epoch: 88 - Loss: 0.6302, Accuracy: 94.70%

Test set: Average loss: 0.5186, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.687906
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.607771
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.604698
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.659635
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.631230
Train Epoch: 89 - Loss: 0.6288, Accuracy: 94.82%

Test set: Average loss: 0.5177, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.614820
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.589412
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.581742
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.659769
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.640752
Train Epoch: 90 - Loss: 0.6288, Accuracy: 94.86%

Test set: Average loss: 0.5183, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.675305
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.577802
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.674688
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.613994
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.622778
Train Epoch: 91 - Loss: 0.6278, Accuracy: 94.85%

Test set: Average loss: 0.5173, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.605237
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.669073
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.574155
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.599307
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.643911
Train Epoch: 92 - Loss: 0.6261, Accuracy: 94.95%

Test set: Average loss: 0.5174, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.667767
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.676977
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.622039
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.648459
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.616763
Train Epoch: 93 - Loss: 0.6244, Accuracy: 94.93%

Test set: Average loss: 0.5172, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.561106
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.600736
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.560338
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.603597
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.612736
Train Epoch: 94 - Loss: 0.6268, Accuracy: 94.89%

Test set: Average loss: 0.5173, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.592051
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.606436
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.668545
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.697134
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.670300
Train Epoch: 95 - Loss: 0.6258, Accuracy: 94.95%

Test set: Average loss: 0.5175, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.652026
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.647214
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.607590
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.631699
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.653442
Train Epoch: 96 - Loss: 0.6241, Accuracy: 94.90%

Test set: Average loss: 0.5170, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.623439
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.656278
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.608735
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.561287
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.626382
Train Epoch: 97 - Loss: 0.6228, Accuracy: 94.97%

Test set: Average loss: 0.5168, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.595703
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.654500
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.629455
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.631269
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.618369
Train Epoch: 98 - Loss: 0.6235, Accuracy: 95.11%

Test set: Average loss: 0.5169, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.667481
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.695748
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.647066
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.702904
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.591343
Train Epoch: 99 - Loss: 0.6254, Accuracy: 94.96%

Test set: Average loss: 0.5165, Accuracy: 9958/10000 (99.58%)

New best model saved with accuracy: 99.58%
Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.648523
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.593645
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.586985
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.607762
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.573788
Train Epoch: 100 - Loss: 0.6241, Accuracy: 95.05%

Test set: Average loss: 0.5167, Accuracy: 9955/10000 (99.55%)


Training completed. Best Accuracy: 99.58%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.106193
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.403303
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.173130
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.018543
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.936510
Train Epoch: 1 - Loss: 1.3345, Accuracy: 69.16%

Test set: Average loss: 0.6832, Accuracy: 9670/10000 (96.70%)

New best model saved with accuracy: 96.70%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 1.055347
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.939361
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.922995
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.846044
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.838379
Train Epoch: 2 - Loss: 0.8775, Accuracy: 85.98%

Test set: Average loss: 0.6037, Accuracy: 9814/10000 (98.14%)

New best model saved with accuracy: 98.14%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.744601
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.808398
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.818540
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.851108
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.785928
Train Epoch: 3 - Loss: 0.8047, Accuracy: 88.65%

Test set: Average loss: 0.5840, Accuracy: 9848/10000 (98.48%)

New best model saved with accuracy: 98.48%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.746155
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.769600
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.698943
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.718332
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.720011
Train Epoch: 4 - Loss: 0.7640, Accuracy: 90.19%

Test set: Average loss: 0.5624, Accuracy: 9895/10000 (98.95%)

New best model saved with accuracy: 98.95%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.710557
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.705168
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.741576
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.679962
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.775724
Train Epoch: 5 - Loss: 0.7446, Accuracy: 90.84%

Test set: Average loss: 0.5610, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.709906
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.742892
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.779689
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.699064
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.708018
Train Epoch: 6 - Loss: 0.7285, Accuracy: 91.44%

Test set: Average loss: 0.5516, Accuracy: 9898/10000 (98.98%)

Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.716704
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.696530
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.716154
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.750030
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.662458
Train Epoch: 7 - Loss: 0.7177, Accuracy: 91.81%

Test set: Average loss: 0.5512, Accuracy: 9895/10000 (98.95%)

Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.702076
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.674975
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.762935
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.763010
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.693831
Train Epoch: 8 - Loss: 0.7089, Accuracy: 92.08%

Test set: Average loss: 0.5471, Accuracy: 9907/10000 (99.07%)

New best model saved with accuracy: 99.07%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.676981
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.685862
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.730329
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.749794
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.673325
Train Epoch: 9 - Loss: 0.6984, Accuracy: 92.51%

Test set: Average loss: 0.5380, Accuracy: 9925/10000 (99.25%)

New best model saved with accuracy: 99.25%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.686315
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.693851
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.720456
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.705394
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.665895
Train Epoch: 10 - Loss: 0.6944, Accuracy: 92.56%

Test set: Average loss: 0.5386, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.719834
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.639194
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.637331
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.689791
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.671044
Train Epoch: 11 - Loss: 0.6906, Accuracy: 92.72%

Test set: Average loss: 0.5391, Accuracy: 9919/10000 (99.19%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.695522
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.733788
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.662935
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.710166
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.700646
Train Epoch: 12 - Loss: 0.6860, Accuracy: 93.00%

Test set: Average loss: 0.5341, Accuracy: 9928/10000 (99.28%)

New best model saved with accuracy: 99.28%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.680722
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.722627
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.710285
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.659637
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.707442
Train Epoch: 13 - Loss: 0.6815, Accuracy: 92.93%

Test set: Average loss: 0.5339, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.629587
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.644673
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.605408
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.648556
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.701791
Train Epoch: 14 - Loss: 0.6778, Accuracy: 93.21%

Test set: Average loss: 0.5336, Accuracy: 9923/10000 (99.23%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.674844
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.643188
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.652821
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.614903
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.662247
Train Epoch: 15 - Loss: 0.6803, Accuracy: 93.09%

Test set: Average loss: 0.5321, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.763951
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.667218
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.690505
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.654877
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.692387
Train Epoch: 16 - Loss: 0.6744, Accuracy: 93.26%

Test set: Average loss: 0.5321, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.657738
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.710704
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.672658
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.686018
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.689362
Train Epoch: 17 - Loss: 0.6709, Accuracy: 93.42%

Test set: Average loss: 0.5299, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.705114
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.670813
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.686102
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.702281
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.718553
Train Epoch: 18 - Loss: 0.6709, Accuracy: 93.52%

Test set: Average loss: 0.5291, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.701488
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.665268
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.629502
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.642628
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.703745
Train Epoch: 19 - Loss: 0.6645, Accuracy: 93.72%

Test set: Average loss: 0.5295, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.648040
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.644327
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.672310
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.675407
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.682071
Train Epoch: 20 - Loss: 0.6646, Accuracy: 93.67%

Test set: Average loss: 0.5289, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.674209
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.587523
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.633070
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.689937
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.707774
Train Epoch: 21 - Loss: 0.6653, Accuracy: 93.62%

Test set: Average loss: 0.5295, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.611818
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.705237
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.632754
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.634762
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.674531
Train Epoch: 22 - Loss: 0.6670, Accuracy: 93.63%

Test set: Average loss: 0.5296, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.699642
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.649378
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.729484
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.684388
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.706455
Train Epoch: 23 - Loss: 0.6635, Accuracy: 93.74%

Test set: Average loss: 0.5290, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.643674
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.717804
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.664617
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.638249
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.617766
Train Epoch: 24 - Loss: 0.6633, Accuracy: 93.75%

Test set: Average loss: 0.5286, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.676556
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.613571
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.607048
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.625250
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.652500
Train Epoch: 25 - Loss: 0.6647, Accuracy: 93.76%

Test set: Average loss: 0.5290, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.701389
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.615657
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.715032
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.709703
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.695371
Train Epoch: 26 - Loss: 0.6774, Accuracy: 93.20%

Test set: Average loss: 0.5360, Accuracy: 9920/10000 (99.20%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.648255
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.746140
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.683402
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.614904
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.638549
Train Epoch: 27 - Loss: 0.6740, Accuracy: 93.24%

Test set: Average loss: 0.5327, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.683956
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.671320
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.643091
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.714729
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.631904
Train Epoch: 28 - Loss: 0.6722, Accuracy: 93.32%

Test set: Average loss: 0.5299, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.688357
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.638429
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.701241
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.657728
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.672135
Train Epoch: 29 - Loss: 0.6680, Accuracy: 93.41%

Test set: Average loss: 0.5293, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.579700
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.664466
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.647827
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.670097
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.649800
Train Epoch: 30 - Loss: 0.6626, Accuracy: 93.72%

Test set: Average loss: 0.5269, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.712311
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.677106
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.708171
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.685144
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.664233
Train Epoch: 31 - Loss: 0.6624, Accuracy: 93.67%

Test set: Average loss: 0.5315, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.678381
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.631472
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.699881
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.701055
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.688964
Train Epoch: 32 - Loss: 0.6609, Accuracy: 93.67%

Test set: Average loss: 0.5258, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.653582
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.723196
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.663869
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.632215
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.656206
Train Epoch: 33 - Loss: 0.6588, Accuracy: 93.78%

Test set: Average loss: 0.5262, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.656922
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.698292
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.704964
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.732678
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.595082
Train Epoch: 34 - Loss: 0.6564, Accuracy: 93.85%

Test set: Average loss: 0.5250, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.650739
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.693717
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.626479
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.744863
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.684577
Train Epoch: 35 - Loss: 0.6560, Accuracy: 93.92%

Test set: Average loss: 0.5251, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.643557
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.681025
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.629139
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.587000
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.654012
Train Epoch: 36 - Loss: 0.6506, Accuracy: 94.10%

Test set: Average loss: 0.5243, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.667024
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.619835
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.673798
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.642186
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.639342
Train Epoch: 37 - Loss: 0.6526, Accuracy: 94.00%

Test set: Average loss: 0.5272, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.655266
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.655280
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.626639
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.608236
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.697105
Train Epoch: 38 - Loss: 0.6475, Accuracy: 94.28%

Test set: Average loss: 0.5241, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.616991
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.676062
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.613595
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.642439
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.647118
Train Epoch: 39 - Loss: 0.6502, Accuracy: 94.12%

Test set: Average loss: 0.5229, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.601168
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.630993
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.657172
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.601818
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.633235
Train Epoch: 40 - Loss: 0.6465, Accuracy: 94.14%

Test set: Average loss: 0.5223, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.703478
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.621312
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.647515
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.664706
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.621829
Train Epoch: 41 - Loss: 0.6449, Accuracy: 94.29%

Test set: Average loss: 0.5226, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.582411
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.643568
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.681265
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.654240
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.640261
Train Epoch: 42 - Loss: 0.6446, Accuracy: 94.35%

Test set: Average loss: 0.5227, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.670166
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.685389
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.622237
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.686248
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.663517
Train Epoch: 43 - Loss: 0.6443, Accuracy: 94.31%

Test set: Average loss: 0.5216, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.621813
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.642006
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.629117
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.669480
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.679190
Train Epoch: 44 - Loss: 0.6410, Accuracy: 94.42%

Test set: Average loss: 0.5216, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.731130
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.605993
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.611375
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.640169
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.645764
Train Epoch: 45 - Loss: 0.6389, Accuracy: 94.53%

Test set: Average loss: 0.5217, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.654097
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.646826
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.617279
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.628394
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.689253
Train Epoch: 46 - Loss: 0.6392, Accuracy: 94.48%

Test set: Average loss: 0.5215, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.654374
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.609564
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.637855
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.576758
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.642339
Train Epoch: 47 - Loss: 0.6383, Accuracy: 94.64%

Test set: Average loss: 0.5216, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.591651
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.618090
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.670011
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.640057
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.648778
Train Epoch: 48 - Loss: 0.6404, Accuracy: 94.44%

Test set: Average loss: 0.5215, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.615695
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.646432
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.611527
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.658766
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.692839
Train Epoch: 49 - Loss: 0.6399, Accuracy: 94.41%

Test set: Average loss: 0.5210, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.636138
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.629846
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.694334
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.640672
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.657818
Train Epoch: 50 - Loss: 0.6392, Accuracy: 94.58%

Test set: Average loss: 0.5212, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.667171
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.664986
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.681548
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.732504
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.770350
Train Epoch: 51 - Loss: 0.6517, Accuracy: 93.99%

Test set: Average loss: 0.5235, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.677997
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.622727
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.633540
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.590211
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.613509
Train Epoch: 52 - Loss: 0.6517, Accuracy: 93.94%

Test set: Average loss: 0.5226, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.646229
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.693525
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.723647
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.701432
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.657785
Train Epoch: 53 - Loss: 0.6485, Accuracy: 94.08%

Test set: Average loss: 0.5233, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.638851
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.659095
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.633764
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.624612
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.613923
Train Epoch: 54 - Loss: 0.6492, Accuracy: 94.00%

Test set: Average loss: 0.5243, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.663086
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.635725
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.689446
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.690013
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.586648
Train Epoch: 55 - Loss: 0.6475, Accuracy: 94.02%

Test set: Average loss: 0.5230, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.640506
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.596392
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.692723
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.658177
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.800369
Train Epoch: 56 - Loss: 0.6435, Accuracy: 94.30%

Test set: Average loss: 0.5217, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.706144
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.629521
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.693484
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.624875
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.670431
Train Epoch: 57 - Loss: 0.6411, Accuracy: 94.41%

Test set: Average loss: 0.5215, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.657401
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.648085
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.620954
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.623516
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.604772
Train Epoch: 58 - Loss: 0.6420, Accuracy: 94.33%

Test set: Average loss: 0.5208, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.640223
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.584335
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.629225
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.650679
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.617645
Train Epoch: 59 - Loss: 0.6429, Accuracy: 94.33%

Test set: Average loss: 0.5202, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.655877
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.607274
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.645822
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.636115
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.635263
Train Epoch: 60 - Loss: 0.6388, Accuracy: 94.41%

Test set: Average loss: 0.5204, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.654251
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.632049
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.658674
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.632526
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.621414
Train Epoch: 61 - Loss: 0.6372, Accuracy: 94.44%

Test set: Average loss: 0.5212, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.650471
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.634033
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.627048
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.634822
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.659091
Train Epoch: 62 - Loss: 0.6354, Accuracy: 94.63%

Test set: Average loss: 0.5205, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.688205
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.612944
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.585901
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.634826
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.604559
Train Epoch: 63 - Loss: 0.6342, Accuracy: 94.65%

Test set: Average loss: 0.5195, Accuracy: 9956/10000 (99.56%)

New best model saved with accuracy: 99.56%
Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.605506
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.578956
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.725803
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.641717
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.590234
Train Epoch: 64 - Loss: 0.6344, Accuracy: 94.68%

Test set: Average loss: 0.5193, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.602197
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.671390
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.641405
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.615542
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.609430
Train Epoch: 65 - Loss: 0.6332, Accuracy: 94.58%

Test set: Average loss: 0.5192, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.679983
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.658345
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.654572
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.623591
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.622951
Train Epoch: 66 - Loss: 0.6338, Accuracy: 94.58%

Test set: Average loss: 0.5190, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.544946
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.635962
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.599835
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.680152
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.620916
Train Epoch: 67 - Loss: 0.6330, Accuracy: 94.69%

Test set: Average loss: 0.5198, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.625966
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.609348
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.627223
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.673163
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.672528
Train Epoch: 68 - Loss: 0.6315, Accuracy: 94.72%

Test set: Average loss: 0.5185, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.656947
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.592034
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.614521
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.661380
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.618389
Train Epoch: 69 - Loss: 0.6305, Accuracy: 94.76%

Test set: Average loss: 0.5184, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.617805
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.628595
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.614051
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.570080
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.592999
Train Epoch: 70 - Loss: 0.6291, Accuracy: 94.88%

Test set: Average loss: 0.5187, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.646196
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.620080
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.658063
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.604382
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.615858
Train Epoch: 71 - Loss: 0.6315, Accuracy: 94.77%

Test set: Average loss: 0.5183, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.638075
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.639976
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.617053
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.577876
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.578193
Train Epoch: 72 - Loss: 0.6293, Accuracy: 94.83%

Test set: Average loss: 0.5181, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.581465
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.710832
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.682061
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.604575
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.580668
Train Epoch: 73 - Loss: 0.6283, Accuracy: 94.89%

Test set: Average loss: 0.5180, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.623064
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.588828
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.590775
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.608772
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.692209
Train Epoch: 74 - Loss: 0.6280, Accuracy: 94.81%

Test set: Average loss: 0.5180, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.618234
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.617928
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.657286
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.587868
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.591606
Train Epoch: 75 - Loss: 0.6269, Accuracy: 94.92%

Test set: Average loss: 0.5181, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.628908
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.644697
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.601296
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.626671
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.572563
Train Epoch: 76 - Loss: 0.6378, Accuracy: 94.52%

Test set: Average loss: 0.5234, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.647377
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.680234
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.656744
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.640578
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.681115
Train Epoch: 77 - Loss: 0.6369, Accuracy: 94.49%

Test set: Average loss: 0.5199, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.618898
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.661018
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.629554
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.654471
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.631715
Train Epoch: 78 - Loss: 0.6356, Accuracy: 94.58%

Test set: Average loss: 0.5205, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.635789
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.637365
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.634134
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.653146
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.648751
Train Epoch: 79 - Loss: 0.6361, Accuracy: 94.56%

Test set: Average loss: 0.5197, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.651736
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.647034
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.656473
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.634639
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.602204
Train Epoch: 80 - Loss: 0.6344, Accuracy: 94.63%

Test set: Average loss: 0.5193, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.673857
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.617043
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.625018
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.669417
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.648689
Train Epoch: 81 - Loss: 0.6361, Accuracy: 94.52%

Test set: Average loss: 0.5200, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.610699
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.640517
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.686624
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.656270
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.649192
Train Epoch: 82 - Loss: 0.6345, Accuracy: 94.55%

Test set: Average loss: 0.5193, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.633592
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.588664
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.655254
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.616183
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.607139
Train Epoch: 83 - Loss: 0.6333, Accuracy: 94.61%

Test set: Average loss: 0.5221, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.647961
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.608919
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.650531
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.743287
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.587802
Train Epoch: 84 - Loss: 0.6330, Accuracy: 94.64%

Test set: Average loss: 0.5188, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.615052
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.629670
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.648859
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.581475
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.591808
Train Epoch: 85 - Loss: 0.6333, Accuracy: 94.71%

Test set: Average loss: 0.5187, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.644003
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.617636
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.674535
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.634274
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.592157
Train Epoch: 86 - Loss: 0.6308, Accuracy: 94.67%

Test set: Average loss: 0.5183, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.608124
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.608958
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.588977
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.714726
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.592265
Train Epoch: 87 - Loss: 0.6287, Accuracy: 94.87%

Test set: Average loss: 0.5179, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.628694
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.623598
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.650340
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.617198
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.633631
Train Epoch: 88 - Loss: 0.6291, Accuracy: 94.79%

Test set: Average loss: 0.5174, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.602959
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.658574
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.623267
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.671885
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.604228
Train Epoch: 89 - Loss: 0.6262, Accuracy: 94.81%

Test set: Average loss: 0.5174, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.650183
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.582198
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.585877
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.563964
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.613872
Train Epoch: 90 - Loss: 0.6257, Accuracy: 94.89%

Test set: Average loss: 0.5167, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.600076
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.630074
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.674580
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.612380
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.605704
Train Epoch: 91 - Loss: 0.6270, Accuracy: 94.82%

Test set: Average loss: 0.5170, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.636058
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.644703
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.578951
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.639986
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.669710
Train Epoch: 92 - Loss: 0.6243, Accuracy: 94.97%

Test set: Average loss: 0.5171, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.643889
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.621129
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.583194
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.590997
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.616528
Train Epoch: 93 - Loss: 0.6263, Accuracy: 94.86%

Test set: Average loss: 0.5170, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.650289
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.606072
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.571154
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.622686
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.585657
Train Epoch: 94 - Loss: 0.6229, Accuracy: 95.10%

Test set: Average loss: 0.5169, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.608266
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.604893
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.688124
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.592875
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.644831
Train Epoch: 95 - Loss: 0.6249, Accuracy: 94.90%

Test set: Average loss: 0.5168, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.563596
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.607211
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.586846
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.605500
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.646032
Train Epoch: 96 - Loss: 0.6218, Accuracy: 95.18%

Test set: Average loss: 0.5169, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.577275
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.599584
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.619545
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.644375
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.637245
Train Epoch: 97 - Loss: 0.6205, Accuracy: 95.17%

Test set: Average loss: 0.5171, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.607359
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.647464
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.622306
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.579989
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.640571
Train Epoch: 98 - Loss: 0.6198, Accuracy: 95.18%

Test set: Average loss: 0.5170, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.646683
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.649316
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.591547
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.619378
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.571228
Train Epoch: 99 - Loss: 0.6197, Accuracy: 95.24%

Test set: Average loss: 0.5166, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.664257
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.623980
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.636445
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.650840
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.607531
Train Epoch: 100 - Loss: 0.6236, Accuracy: 94.99%

Test set: Average loss: 0.5168, Accuracy: 9955/10000 (99.55%)


Training completed. Best Accuracy: 99.56%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.046580
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.372719
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.039375
Train Epoch: 1 [38400/60000 (64%)]	Loss: 0.874133
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.818793
Train Epoch: 1 - Loss: 1.3073, Accuracy: 70.92%

Test set: Average loss: 0.6688, Accuracy: 9679/10000 (96.79%)

New best model saved with accuracy: 96.79%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.967997
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.866136
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.926119
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.829539
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.902012
Train Epoch: 2 - Loss: 0.8776, Accuracy: 85.67%

Test set: Average loss: 0.6060, Accuracy: 9784/10000 (97.84%)

New best model saved with accuracy: 97.84%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.882258
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.884801
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.792578
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.790940
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.708361
Train Epoch: 3 - Loss: 0.8087, Accuracy: 88.41%

Test set: Average loss: 0.5844, Accuracy: 9828/10000 (98.28%)

New best model saved with accuracy: 98.28%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.806248
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.786466
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.727211
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.815750
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.689015
Train Epoch: 4 - Loss: 0.7709, Accuracy: 89.84%

Test set: Average loss: 0.5706, Accuracy: 9862/10000 (98.62%)

New best model saved with accuracy: 98.62%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.760760
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.733027
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.652503
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.673973
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.689671
Train Epoch: 5 - Loss: 0.7520, Accuracy: 90.56%

Test set: Average loss: 0.5615, Accuracy: 9876/10000 (98.76%)

New best model saved with accuracy: 98.76%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.754346
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.739299
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.707305
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.696565
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.686593
Train Epoch: 6 - Loss: 0.7358, Accuracy: 91.11%

Test set: Average loss: 0.5564, Accuracy: 9879/10000 (98.79%)

New best model saved with accuracy: 98.79%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.726033
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.712475
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.723483
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.732949
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.689215
Train Epoch: 7 - Loss: 0.7250, Accuracy: 91.38%

Test set: Average loss: 0.5535, Accuracy: 9884/10000 (98.84%)

New best model saved with accuracy: 98.84%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.636815
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.735521
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.669665
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.700067
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.710373
Train Epoch: 8 - Loss: 0.7172, Accuracy: 91.66%

Test set: Average loss: 0.5468, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.701717
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.673201
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.657520
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.666509
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.702444
Train Epoch: 9 - Loss: 0.7111, Accuracy: 91.86%

Test set: Average loss: 0.5420, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.693928
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.755718
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.760454
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.693618
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.679914
Train Epoch: 10 - Loss: 0.7021, Accuracy: 92.40%

Test set: Average loss: 0.5414, Accuracy: 9908/10000 (99.08%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.709120
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.597324
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.694060
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.682822
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.643189
Train Epoch: 11 - Loss: 0.6976, Accuracy: 92.47%

Test set: Average loss: 0.5384, Accuracy: 9914/10000 (99.14%)

New best model saved with accuracy: 99.14%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.680891
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.666018
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.787481
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.676358
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.679186
Train Epoch: 12 - Loss: 0.6923, Accuracy: 92.64%

Test set: Average loss: 0.5410, Accuracy: 9913/10000 (99.13%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.659278
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.624674
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.699451
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.625023
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.729138
Train Epoch: 13 - Loss: 0.6902, Accuracy: 92.70%

Test set: Average loss: 0.5381, Accuracy: 9916/10000 (99.16%)

New best model saved with accuracy: 99.16%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.629799
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.662479
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.732585
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.641434
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.662466
Train Epoch: 14 - Loss: 0.6838, Accuracy: 92.96%

Test set: Average loss: 0.5343, Accuracy: 9914/10000 (99.14%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.706820
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.677005
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.662050
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.665058
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.725896
Train Epoch: 15 - Loss: 0.6793, Accuracy: 93.12%

Test set: Average loss: 0.5336, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.747813
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.671710
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.719628
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.665745
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.728937
Train Epoch: 16 - Loss: 0.6804, Accuracy: 93.05%

Test set: Average loss: 0.5338, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.659240
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.630385
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.670593
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.684251
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.625938
Train Epoch: 17 - Loss: 0.6778, Accuracy: 93.05%

Test set: Average loss: 0.5315, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.620845
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.636265
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.656216
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.704858
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.765613
Train Epoch: 18 - Loss: 0.6755, Accuracy: 93.19%

Test set: Average loss: 0.5310, Accuracy: 9932/10000 (99.32%)

New best model saved with accuracy: 99.32%
Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.679867
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.667412
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.708992
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.647090
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.697045
Train Epoch: 19 - Loss: 0.6740, Accuracy: 93.29%

Test set: Average loss: 0.5313, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.699044
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.631389
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.704609
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.669409
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.650544
Train Epoch: 20 - Loss: 0.6708, Accuracy: 93.49%

Test set: Average loss: 0.5308, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.667432
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.642523
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.656325
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.650228
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.646140
Train Epoch: 21 - Loss: 0.6723, Accuracy: 93.41%

Test set: Average loss: 0.5311, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.693144
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.650879
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.636715
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.696864
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.700041
Train Epoch: 22 - Loss: 0.6723, Accuracy: 93.36%

Test set: Average loss: 0.5309, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.651584
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.657242
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.687395
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.651980
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.663065
Train Epoch: 23 - Loss: 0.6712, Accuracy: 93.45%

Test set: Average loss: 0.5308, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.673954
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.702386
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.648982
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.690614
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.689773
Train Epoch: 24 - Loss: 0.6691, Accuracy: 93.53%

Test set: Average loss: 0.5304, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.658340
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.648642
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.635510
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.644783
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.673596
Train Epoch: 25 - Loss: 0.6702, Accuracy: 93.51%

Test set: Average loss: 0.5302, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.698601
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.657877
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.623524
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.710368
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.758554
Train Epoch: 26 - Loss: 0.6852, Accuracy: 92.92%

Test set: Average loss: 0.5341, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.687397
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.685778
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.665150
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.614302
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.693825
Train Epoch: 27 - Loss: 0.6786, Accuracy: 93.09%

Test set: Average loss: 0.5327, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.746652
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.667531
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.669647
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.701241
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.587174
Train Epoch: 28 - Loss: 0.6735, Accuracy: 93.34%

Test set: Average loss: 0.5306, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.655747
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.590322
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.634052
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.611616
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.700694
Train Epoch: 29 - Loss: 0.6763, Accuracy: 93.10%

Test set: Average loss: 0.5311, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.721181
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.746334
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.665880
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.713788
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.664153
Train Epoch: 30 - Loss: 0.6722, Accuracy: 93.25%

Test set: Average loss: 0.5347, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.676647
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.721387
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.637392
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.647556
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.716131
Train Epoch: 31 - Loss: 0.6669, Accuracy: 93.51%

Test set: Average loss: 0.5281, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.734624
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.656863
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.658750
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.631300
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.618928
Train Epoch: 32 - Loss: 0.6666, Accuracy: 93.46%

Test set: Average loss: 0.5314, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.661840
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.681532
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.631788
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.623838
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.733770
Train Epoch: 33 - Loss: 0.6614, Accuracy: 93.67%

Test set: Average loss: 0.5296, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.656815
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.680208
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.654162
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.682728
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.668080
Train Epoch: 34 - Loss: 0.6617, Accuracy: 93.59%

Test set: Average loss: 0.5269, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.632643
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.712321
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.669988
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.679651
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.659939
Train Epoch: 35 - Loss: 0.6584, Accuracy: 93.89%

Test set: Average loss: 0.5282, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.615445
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.691469
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.662145
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.609442
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.691462
Train Epoch: 36 - Loss: 0.6540, Accuracy: 93.97%

Test set: Average loss: 0.5241, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.601105
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.625027
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.648439
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.683785
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.693538
Train Epoch: 37 - Loss: 0.6546, Accuracy: 93.88%

Test set: Average loss: 0.5265, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.575132
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.679311
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.627667
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.610574
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.677924
Train Epoch: 38 - Loss: 0.6536, Accuracy: 93.96%

Test set: Average loss: 0.5249, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.665430
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.790931
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.672091
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.600680
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.675093
Train Epoch: 39 - Loss: 0.6522, Accuracy: 93.99%

Test set: Average loss: 0.5245, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.614525
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.660724
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.635255
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.587722
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.575773
Train Epoch: 40 - Loss: 0.6514, Accuracy: 93.94%

Test set: Average loss: 0.5241, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.647684
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.605024
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.648204
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.665108
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.705054
Train Epoch: 41 - Loss: 0.6511, Accuracy: 94.00%

Test set: Average loss: 0.5236, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.660388
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.696536
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.628229
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.659921
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.683234
Train Epoch: 42 - Loss: 0.6483, Accuracy: 94.19%

Test set: Average loss: 0.5225, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.654441
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.595643
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.577166
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.595237
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.600523
Train Epoch: 43 - Loss: 0.6456, Accuracy: 94.29%

Test set: Average loss: 0.5232, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.625300
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.703133
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.644548
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.662620
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.653475
Train Epoch: 44 - Loss: 0.6449, Accuracy: 94.27%

Test set: Average loss: 0.5222, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.635255
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.676558
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.714816
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.611906
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.691460
Train Epoch: 45 - Loss: 0.6478, Accuracy: 94.13%

Test set: Average loss: 0.5223, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.641201
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.684369
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.611152
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.613351
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.619429
Train Epoch: 46 - Loss: 0.6433, Accuracy: 94.38%

Test set: Average loss: 0.5224, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.621059
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.652312
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.645544
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.612578
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.615651
Train Epoch: 47 - Loss: 0.6448, Accuracy: 94.35%

Test set: Average loss: 0.5223, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.659534
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.665802
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.667589
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.647330
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.632652
Train Epoch: 48 - Loss: 0.6443, Accuracy: 94.31%

Test set: Average loss: 0.5218, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.652023
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.692607
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.710264
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.586219
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.679655
Train Epoch: 49 - Loss: 0.6411, Accuracy: 94.51%

Test set: Average loss: 0.5218, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.594098
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.628351
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.720579
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.618648
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.678178
Train Epoch: 50 - Loss: 0.6450, Accuracy: 94.22%

Test set: Average loss: 0.5221, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.649290
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.650347
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.622060
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.672519
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.698529
Train Epoch: 51 - Loss: 0.6548, Accuracy: 93.89%

Test set: Average loss: 0.5247, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.688810
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.631666
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.627728
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.581555
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.629609
Train Epoch: 52 - Loss: 0.6515, Accuracy: 94.03%

Test set: Average loss: 0.5236, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.624415
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.634639
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.662913
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.696854
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.626769
Train Epoch: 53 - Loss: 0.6536, Accuracy: 93.92%

Test set: Average loss: 0.5238, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.615036
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.681535
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.683205
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.686446
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.724517
Train Epoch: 54 - Loss: 0.6518, Accuracy: 93.91%

Test set: Average loss: 0.5242, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.639462
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.659966
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.636557
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.606880
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.627622
Train Epoch: 55 - Loss: 0.6484, Accuracy: 94.16%

Test set: Average loss: 0.5242, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.650698
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.629901
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.591684
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.647106
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.658474
Train Epoch: 56 - Loss: 0.6460, Accuracy: 94.18%

Test set: Average loss: 0.5218, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.687455
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.616400
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.639519
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.664106
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.632955
Train Epoch: 57 - Loss: 0.6471, Accuracy: 94.16%

Test set: Average loss: 0.5215, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.672312
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.646693
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.609883
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.637899
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.638594
Train Epoch: 58 - Loss: 0.6435, Accuracy: 94.27%

Test set: Average loss: 0.5217, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.588006
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.612787
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.628942
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.679375
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.601773
Train Epoch: 59 - Loss: 0.6424, Accuracy: 94.28%

Test set: Average loss: 0.5215, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.581165
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.634670
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.684440
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.633769
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.628027
Train Epoch: 60 - Loss: 0.6397, Accuracy: 94.38%

Test set: Average loss: 0.5206, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.633466
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.636501
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.596968
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.576386
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.597268
Train Epoch: 61 - Loss: 0.6385, Accuracy: 94.53%

Test set: Average loss: 0.5200, Accuracy: 9953/10000 (99.53%)

New best model saved with accuracy: 99.53%
Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.650559
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.602428
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.598261
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.624571
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.690378
Train Epoch: 62 - Loss: 0.6402, Accuracy: 94.36%

Test set: Average loss: 0.5198, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.609038
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.614307
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.627858
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.643999
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.570185
Train Epoch: 63 - Loss: 0.6399, Accuracy: 94.42%

Test set: Average loss: 0.5213, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.624948
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.613353
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.589549
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.652328
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.645747
Train Epoch: 64 - Loss: 0.6392, Accuracy: 94.47%

Test set: Average loss: 0.5203, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.659294
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.630461
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.580310
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.675751
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.573083
Train Epoch: 65 - Loss: 0.6371, Accuracy: 94.51%

Test set: Average loss: 0.5193, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.581286
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.702545
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.662037
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.598072
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.582764
Train Epoch: 66 - Loss: 0.6331, Accuracy: 94.66%

Test set: Average loss: 0.5192, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.606025
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.736824
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.597569
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.619626
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.610176
Train Epoch: 67 - Loss: 0.6336, Accuracy: 94.73%

Test set: Average loss: 0.5190, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.700053
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.626295
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.675365
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.602488
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.649462
Train Epoch: 68 - Loss: 0.6356, Accuracy: 94.53%

Test set: Average loss: 0.5188, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.640640
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.639957
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.640959
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.603454
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.619808
Train Epoch: 69 - Loss: 0.6340, Accuracy: 94.62%

Test set: Average loss: 0.5187, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.642032
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.601816
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.583404
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.621850
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.696575
Train Epoch: 70 - Loss: 0.6318, Accuracy: 94.79%

Test set: Average loss: 0.5186, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.606060
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.601327
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.652920
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.643378
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.650176
Train Epoch: 71 - Loss: 0.6320, Accuracy: 94.66%

Test set: Average loss: 0.5186, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.616709
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.591559
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.643100
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.609008
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.660162
Train Epoch: 72 - Loss: 0.6312, Accuracy: 94.74%

Test set: Average loss: 0.5183, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.636557
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.632283
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.671118
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.641000
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.662925
Train Epoch: 73 - Loss: 0.6312, Accuracy: 94.75%

Test set: Average loss: 0.5184, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.614732
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.646073
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.591239
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.645833
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.599434
Train Epoch: 74 - Loss: 0.6305, Accuracy: 94.80%

Test set: Average loss: 0.5186, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.620959
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.690576
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.640692
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.599682
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.631645
Train Epoch: 75 - Loss: 0.6296, Accuracy: 94.85%

Test set: Average loss: 0.5184, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.586372
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.691879
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.637008
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.604968
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.647337
Train Epoch: 76 - Loss: 0.6402, Accuracy: 94.42%

Test set: Average loss: 0.5198, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.726117
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.639192
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.689131
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.579953
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.568914
Train Epoch: 77 - Loss: 0.6425, Accuracy: 94.31%

Test set: Average loss: 0.5200, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.633238
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.675930
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.654407
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.668672
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.636921
Train Epoch: 78 - Loss: 0.6386, Accuracy: 94.45%

Test set: Average loss: 0.5193, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.639069
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.646935
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.650716
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.627639
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.619206
Train Epoch: 79 - Loss: 0.6358, Accuracy: 94.57%

Test set: Average loss: 0.5208, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.642939
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.641713
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.621324
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.625859
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.643190
Train Epoch: 80 - Loss: 0.6379, Accuracy: 94.47%

Test set: Average loss: 0.5193, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.625863
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.600816
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.702437
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.621763
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.688347
Train Epoch: 81 - Loss: 0.6360, Accuracy: 94.54%

Test set: Average loss: 0.5199, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.635515
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.604523
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.601250
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.623253
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.621133
Train Epoch: 82 - Loss: 0.6345, Accuracy: 94.64%

Test set: Average loss: 0.5206, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.639076
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.595503
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.654272
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.640060
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.664459
Train Epoch: 83 - Loss: 0.6338, Accuracy: 94.64%

Test set: Average loss: 0.5199, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.603880
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.669848
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.607857
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.611419
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.650296
Train Epoch: 84 - Loss: 0.6329, Accuracy: 94.67%

Test set: Average loss: 0.5185, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.580675
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.673464
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.624586
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.580522
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.625048
Train Epoch: 85 - Loss: 0.6318, Accuracy: 94.79%

Test set: Average loss: 0.5180, Accuracy: 9955/10000 (99.55%)

New best model saved with accuracy: 99.55%
Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.618781
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.615383
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.662362
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.622240
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.596166
Train Epoch: 86 - Loss: 0.6320, Accuracy: 94.72%

Test set: Average loss: 0.5178, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.612312
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.651291
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.630567
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.672907
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.638116
Train Epoch: 87 - Loss: 0.6298, Accuracy: 94.79%

Test set: Average loss: 0.5183, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.643194
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.606881
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.645664
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.648551
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.664134
Train Epoch: 88 - Loss: 0.6304, Accuracy: 94.73%

Test set: Average loss: 0.5168, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.661908
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.630562
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.660039
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.595122
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.592102
Train Epoch: 89 - Loss: 0.6301, Accuracy: 94.75%

Test set: Average loss: 0.5169, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.602971
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.571515
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.611097
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.632677
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.671425
Train Epoch: 90 - Loss: 0.6273, Accuracy: 94.87%

Test set: Average loss: 0.5160, Accuracy: 9958/10000 (99.58%)

New best model saved with accuracy: 99.58%
Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.592743
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.586787
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.604443
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.647756
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.597757
Train Epoch: 91 - Loss: 0.6268, Accuracy: 94.90%

Test set: Average loss: 0.5168, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.605639
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.603490
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.606191
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.663441
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.580467
Train Epoch: 92 - Loss: 0.6251, Accuracy: 95.00%

Test set: Average loss: 0.5168, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.580983
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.597518
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.596816
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.662764
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.668835
Train Epoch: 93 - Loss: 0.6255, Accuracy: 94.98%

Test set: Average loss: 0.5162, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.684926
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.614642
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.662716
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.616743
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.592565
Train Epoch: 94 - Loss: 0.6254, Accuracy: 95.01%

Test set: Average loss: 0.5162, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.612264
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.611928
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.620652
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.585808
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.627555
Train Epoch: 95 - Loss: 0.6230, Accuracy: 95.00%

Test set: Average loss: 0.5161, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.640410
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.685999
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.667459
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.606419
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.634006
Train Epoch: 96 - Loss: 0.6231, Accuracy: 95.09%

Test set: Average loss: 0.5162, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.618392
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.658104
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.651938
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.646944
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.582167
Train Epoch: 97 - Loss: 0.6260, Accuracy: 94.89%

Test set: Average loss: 0.5161, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.584891
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.706160
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.622971
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.581923
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.645294
Train Epoch: 98 - Loss: 0.6203, Accuracy: 95.17%

Test set: Average loss: 0.5159, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.630237
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.619089
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.616956
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.590180
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.587931
Train Epoch: 99 - Loss: 0.6219, Accuracy: 95.14%

Test set: Average loss: 0.5160, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.669859
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.673586
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.649886
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.614312
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.606420
Train Epoch: 100 - Loss: 0.6219, Accuracy: 95.06%

Test set: Average loss: 0.5162, Accuracy: 9951/10000 (99.51%)


Training completed. Best Accuracy: 99.58%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.213941
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.505388
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.173237
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.126925
Train Epoch: 1 [51200/60000 (85%)]	Loss: 1.078813
Train Epoch: 1 - Loss: 1.3324, Accuracy: 68.89%

Test set: Average loss: 0.6824, Accuracy: 9667/10000 (96.67%)

New best model saved with accuracy: 96.67%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 1.001965
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.878215
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.938965
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.856393
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.812997
Train Epoch: 2 - Loss: 0.8930, Accuracy: 85.19%

Test set: Average loss: 0.6047, Accuracy: 9834/10000 (98.34%)

New best model saved with accuracy: 98.34%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.809178
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.802193
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.842672
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.846184
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.727637
Train Epoch: 3 - Loss: 0.8142, Accuracy: 88.22%

Test set: Average loss: 0.5828, Accuracy: 9857/10000 (98.57%)

New best model saved with accuracy: 98.57%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.880866
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.708393
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.771186
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.878707
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.800304
Train Epoch: 4 - Loss: 0.7728, Accuracy: 89.83%

Test set: Average loss: 0.5668, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.792941
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.759591
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.750905
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.718988
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.906923
Train Epoch: 5 - Loss: 0.7564, Accuracy: 90.45%

Test set: Average loss: 0.5598, Accuracy: 9893/10000 (98.93%)

New best model saved with accuracy: 98.93%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.793004
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.804355
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.744893
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.775789
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.708760
Train Epoch: 6 - Loss: 0.7374, Accuracy: 91.06%

Test set: Average loss: 0.5535, Accuracy: 9900/10000 (99.00%)

New best model saved with accuracy: 99.00%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.698869
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.740496
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.794701
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.701263
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.711060
Train Epoch: 7 - Loss: 0.7266, Accuracy: 91.42%

Test set: Average loss: 0.5489, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.688093
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.665882
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.735887
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.685848
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.701196
Train Epoch: 8 - Loss: 0.7145, Accuracy: 91.94%

Test set: Average loss: 0.5464, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.718480
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.731313
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.671044
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.623510
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.867701
Train Epoch: 9 - Loss: 0.7051, Accuracy: 92.23%

Test set: Average loss: 0.5441, Accuracy: 9909/10000 (99.09%)

Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.686025
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.680398
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.732478
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.703125
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.678014
Train Epoch: 10 - Loss: 0.7009, Accuracy: 92.38%

Test set: Average loss: 0.5405, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.686228
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.717659
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.706128
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.669772
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.686523
Train Epoch: 11 - Loss: 0.6949, Accuracy: 92.42%

Test set: Average loss: 0.5385, Accuracy: 9928/10000 (99.28%)

New best model saved with accuracy: 99.28%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.735385
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.716080
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.788838
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.699652
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.651961
Train Epoch: 12 - Loss: 0.6924, Accuracy: 92.64%

Test set: Average loss: 0.5374, Accuracy: 9933/10000 (99.33%)

New best model saved with accuracy: 99.33%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.716917
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.673007
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.601244
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.672179
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.624756
Train Epoch: 13 - Loss: 0.6900, Accuracy: 92.72%

Test set: Average loss: 0.5354, Accuracy: 9939/10000 (99.39%)

New best model saved with accuracy: 99.39%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.663739
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.655985
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.705218
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.684191
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.691839
Train Epoch: 14 - Loss: 0.6842, Accuracy: 92.85%

Test set: Average loss: 0.5356, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.687841
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.669139
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.674772
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.641719
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.726799
Train Epoch: 15 - Loss: 0.6845, Accuracy: 92.83%

Test set: Average loss: 0.5346, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.713436
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.669651
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.644871
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.670043
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.697249
Train Epoch: 16 - Loss: 0.6786, Accuracy: 93.22%

Test set: Average loss: 0.5332, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.680372
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.664520
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.761202
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.710998
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.661179
Train Epoch: 17 - Loss: 0.6784, Accuracy: 93.20%

Test set: Average loss: 0.5341, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.647074
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.682035
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.663450
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.670959
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.659035
Train Epoch: 18 - Loss: 0.6776, Accuracy: 93.15%

Test set: Average loss: 0.5320, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.648629
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.718742
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.662853
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.716141
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.621218
Train Epoch: 19 - Loss: 0.6744, Accuracy: 93.20%

Test set: Average loss: 0.5316, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.711452
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.707763
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.670007
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.688594
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.681901
Train Epoch: 20 - Loss: 0.6735, Accuracy: 93.41%

Test set: Average loss: 0.5314, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.651616
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.656500
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.642578
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.665996
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.652581
Train Epoch: 21 - Loss: 0.6725, Accuracy: 93.42%

Test set: Average loss: 0.5316, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.634859
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.681962
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.661296
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.720907
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.753770
Train Epoch: 22 - Loss: 0.6723, Accuracy: 93.41%

Test set: Average loss: 0.5310, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.675466
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.633771
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.661367
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.650627
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.672326
Train Epoch: 23 - Loss: 0.6705, Accuracy: 93.44%

Test set: Average loss: 0.5313, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.618282
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.640702
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.698140
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.714144
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.660186
Train Epoch: 24 - Loss: 0.6693, Accuracy: 93.45%

Test set: Average loss: 0.5311, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.636377
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.693942
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.729473
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.691461
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.633860
Train Epoch: 25 - Loss: 0.6732, Accuracy: 93.32%

Test set: Average loss: 0.5306, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.635952
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.619465
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.664704
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.661024
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.720670
Train Epoch: 26 - Loss: 0.6822, Accuracy: 93.00%

Test set: Average loss: 0.5340, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.702208
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.713310
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.681929
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.627193
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.715727
Train Epoch: 27 - Loss: 0.6805, Accuracy: 93.09%

Test set: Average loss: 0.5322, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.592075
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.679859
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.672206
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.705256
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.679748
Train Epoch: 28 - Loss: 0.6754, Accuracy: 93.23%

Test set: Average loss: 0.5326, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.703832
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.647833
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.675120
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.664350
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.727023
Train Epoch: 29 - Loss: 0.6740, Accuracy: 93.31%

Test set: Average loss: 0.5329, Accuracy: 9926/10000 (99.26%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.695475
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.625103
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.755924
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.610255
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.716164
Train Epoch: 30 - Loss: 0.6707, Accuracy: 93.31%

Test set: Average loss: 0.5320, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.672809
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.688566
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.721068
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.691794
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.690010
Train Epoch: 31 - Loss: 0.6716, Accuracy: 93.23%

Test set: Average loss: 0.5284, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.637673
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.660756
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.691921
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.700390
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.646793
Train Epoch: 32 - Loss: 0.6665, Accuracy: 93.53%

Test set: Average loss: 0.5277, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.657463
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.652468
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.749480
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.668583
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.675117
Train Epoch: 33 - Loss: 0.6660, Accuracy: 93.47%

Test set: Average loss: 0.5280, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.656550
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.686180
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.624685
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.643671
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.613429
Train Epoch: 34 - Loss: 0.6639, Accuracy: 93.53%

Test set: Average loss: 0.5267, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.712050
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.646111
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.609000
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.645367
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.647504
Train Epoch: 35 - Loss: 0.6564, Accuracy: 93.89%

Test set: Average loss: 0.5267, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.639100
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.664918
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.625103
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.682180
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.591021
Train Epoch: 36 - Loss: 0.6584, Accuracy: 93.74%

Test set: Average loss: 0.5268, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.652562
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.624502
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.727125
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.636577
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.612621
Train Epoch: 37 - Loss: 0.6566, Accuracy: 93.99%

Test set: Average loss: 0.5258, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.630445
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.620416
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.593753
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.686173
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.607627
Train Epoch: 38 - Loss: 0.6542, Accuracy: 93.99%

Test set: Average loss: 0.5245, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.687640
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.623207
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.697276
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.672308
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.733697
Train Epoch: 39 - Loss: 0.6546, Accuracy: 93.93%

Test set: Average loss: 0.5248, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.622159
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.643136
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.609820
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.594869
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.619490
Train Epoch: 40 - Loss: 0.6508, Accuracy: 94.05%

Test set: Average loss: 0.5246, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.591744
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.676273
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.661011
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.664163
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.645311
Train Epoch: 41 - Loss: 0.6498, Accuracy: 94.03%

Test set: Average loss: 0.5244, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.588996
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.623465
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.667737
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.662449
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.655377
Train Epoch: 42 - Loss: 0.6481, Accuracy: 94.16%

Test set: Average loss: 0.5237, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.709786
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.632795
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.626690
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.741885
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.682494
Train Epoch: 43 - Loss: 0.6480, Accuracy: 94.25%

Test set: Average loss: 0.5240, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.644371
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.687525
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.616615
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.638186
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.667270
Train Epoch: 44 - Loss: 0.6446, Accuracy: 94.31%

Test set: Average loss: 0.5233, Accuracy: 9954/10000 (99.54%)

New best model saved with accuracy: 99.54%
Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.646857
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.584082
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.678182
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.638517
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.674118
Train Epoch: 45 - Loss: 0.6447, Accuracy: 94.36%

Test set: Average loss: 0.5231, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.651675
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.655575
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.658031
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.619885
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.672722
Train Epoch: 46 - Loss: 0.6454, Accuracy: 94.23%

Test set: Average loss: 0.5227, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.666650
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.643242
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.659090
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.670235
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.620979
Train Epoch: 47 - Loss: 0.6431, Accuracy: 94.36%

Test set: Average loss: 0.5235, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.671916
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.662272
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.685794
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.636826
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.667183
Train Epoch: 48 - Loss: 0.6442, Accuracy: 94.37%

Test set: Average loss: 0.5229, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.677188
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.673948
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.612933
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.666155
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.660902
Train Epoch: 49 - Loss: 0.6448, Accuracy: 94.33%

Test set: Average loss: 0.5230, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.625819
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.567742
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.684349
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.680580
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.611902
Train Epoch: 50 - Loss: 0.6445, Accuracy: 94.38%

Test set: Average loss: 0.5229, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.653477
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.652187
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.678791
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.665316
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.675966
Train Epoch: 51 - Loss: 0.6540, Accuracy: 93.91%

Test set: Average loss: 0.5265, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.672791
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.679591
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.617947
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.668728
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.751413
Train Epoch: 52 - Loss: 0.6551, Accuracy: 93.86%

Test set: Average loss: 0.5235, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.653516
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.601402
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.618523
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.658673
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.662462
Train Epoch: 53 - Loss: 0.6524, Accuracy: 94.05%

Test set: Average loss: 0.5252, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.612939
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.696522
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.734854
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.606431
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.655851
Train Epoch: 54 - Loss: 0.6503, Accuracy: 94.00%

Test set: Average loss: 0.5235, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.652242
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.656575
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.577738
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.652404
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.614374
Train Epoch: 55 - Loss: 0.6500, Accuracy: 94.13%

Test set: Average loss: 0.5243, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.664396
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.674057
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.666072
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.610683
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.693504
Train Epoch: 56 - Loss: 0.6504, Accuracy: 94.03%

Test set: Average loss: 0.5225, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.679762
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.594716
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.690223
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.642738
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.697326
Train Epoch: 57 - Loss: 0.6441, Accuracy: 94.19%

Test set: Average loss: 0.5223, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.628181
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.678524
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.599401
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.741550
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.664503
Train Epoch: 58 - Loss: 0.6457, Accuracy: 94.22%

Test set: Average loss: 0.5223, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.632033
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.655017
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.618461
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.688126
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.606302
Train Epoch: 59 - Loss: 0.6462, Accuracy: 94.16%

Test set: Average loss: 0.5223, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.643390
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.636789
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.597139
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.665149
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.643272
Train Epoch: 60 - Loss: 0.6465, Accuracy: 94.10%

Test set: Average loss: 0.5212, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.652369
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.588612
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.651069
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.658547
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.688614
Train Epoch: 61 - Loss: 0.6412, Accuracy: 94.37%

Test set: Average loss: 0.5209, Accuracy: 9956/10000 (99.56%)

New best model saved with accuracy: 99.56%
Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.593086
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.666378
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.637822
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.696610
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.631931
Train Epoch: 62 - Loss: 0.6420, Accuracy: 94.32%

Test set: Average loss: 0.5213, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.615907
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.666754
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.670531
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.630959
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.706944
Train Epoch: 63 - Loss: 0.6413, Accuracy: 94.33%

Test set: Average loss: 0.5216, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.600517
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.661169
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.636272
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.648344
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.630614
Train Epoch: 64 - Loss: 0.6394, Accuracy: 94.47%

Test set: Average loss: 0.5203, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.639168
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.747757
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.644993
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.687527
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.683734
Train Epoch: 65 - Loss: 0.6374, Accuracy: 94.48%

Test set: Average loss: 0.5200, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.619876
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.627150
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.649862
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.660603
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.681398
Train Epoch: 66 - Loss: 0.6371, Accuracy: 94.56%

Test set: Average loss: 0.5197, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.640449
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.654806
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.583306
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.630637
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.620883
Train Epoch: 67 - Loss: 0.6355, Accuracy: 94.73%

Test set: Average loss: 0.5197, Accuracy: 9958/10000 (99.58%)

New best model saved with accuracy: 99.58%
Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.674102
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.706743
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.639050
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.646607
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.679072
Train Epoch: 68 - Loss: 0.6351, Accuracy: 94.64%

Test set: Average loss: 0.5196, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.679263
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.638226
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.651425
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.585730
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.606406
Train Epoch: 69 - Loss: 0.6318, Accuracy: 94.70%

Test set: Average loss: 0.5193, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.706502
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.614050
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.586519
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.603603
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.610466
Train Epoch: 70 - Loss: 0.6340, Accuracy: 94.67%

Test set: Average loss: 0.5190, Accuracy: 9960/10000 (99.60%)

New best model saved with accuracy: 99.60%
Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.629368
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.661320
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.645054
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.615945
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.584485
Train Epoch: 71 - Loss: 0.6342, Accuracy: 94.69%

Test set: Average loss: 0.5193, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.591150
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.622546
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.581802
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.566527
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.644161
Train Epoch: 72 - Loss: 0.6331, Accuracy: 94.80%

Test set: Average loss: 0.5194, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.596511
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.593500
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.647185
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.600508
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.617179
Train Epoch: 73 - Loss: 0.6351, Accuracy: 94.59%

Test set: Average loss: 0.5190, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.658162
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.600311
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.633546
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.686845
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.602175
Train Epoch: 74 - Loss: 0.6300, Accuracy: 94.80%

Test set: Average loss: 0.5191, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.689022
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.643259
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.630326
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.686635
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.623562
Train Epoch: 75 - Loss: 0.6336, Accuracy: 94.67%

Test set: Average loss: 0.5190, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.648915
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.648903
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.620086
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.635772
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.659057
Train Epoch: 76 - Loss: 0.6425, Accuracy: 94.22%

Test set: Average loss: 0.5209, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.693815
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.592129
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.619316
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.666365
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.590067
Train Epoch: 77 - Loss: 0.6382, Accuracy: 94.49%

Test set: Average loss: 0.5203, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.668752
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.634235
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.622489
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.699882
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.667920
Train Epoch: 78 - Loss: 0.6416, Accuracy: 94.38%

Test set: Average loss: 0.5200, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.623343
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.684954
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.619965
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.650044
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.630807
Train Epoch: 79 - Loss: 0.6401, Accuracy: 94.39%

Test set: Average loss: 0.5199, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.647902
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.684789
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.678641
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.655863
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.642230
Train Epoch: 80 - Loss: 0.6413, Accuracy: 94.32%

Test set: Average loss: 0.5215, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.635249
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.599039
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.673299
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.696873
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.585404
Train Epoch: 81 - Loss: 0.6391, Accuracy: 94.45%

Test set: Average loss: 0.5228, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.666834
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.590336
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.602144
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.636669
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.667143
Train Epoch: 82 - Loss: 0.6356, Accuracy: 94.59%

Test set: Average loss: 0.5190, Accuracy: 9961/10000 (99.61%)

New best model saved with accuracy: 99.61%
Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.621059
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.726310
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.603230
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.626867
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.597774
Train Epoch: 83 - Loss: 0.6404, Accuracy: 94.37%

Test set: Average loss: 0.5198, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.618236
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.591675
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.608754
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.615449
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.632392
Train Epoch: 84 - Loss: 0.6330, Accuracy: 94.68%

Test set: Average loss: 0.5184, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.625029
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.665574
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.661666
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.646289
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.632527
Train Epoch: 85 - Loss: 0.6338, Accuracy: 94.49%

Test set: Average loss: 0.5189, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.620453
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.629972
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.709378
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.664107
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.585717
Train Epoch: 86 - Loss: 0.6344, Accuracy: 94.56%

Test set: Average loss: 0.5182, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.647523
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.666359
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.604756
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.617241
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.669756
Train Epoch: 87 - Loss: 0.6320, Accuracy: 94.75%

Test set: Average loss: 0.5183, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.612091
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.661888
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.583409
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.575278
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.651171
Train Epoch: 88 - Loss: 0.6277, Accuracy: 94.97%

Test set: Average loss: 0.5176, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.646266
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.613433
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.660888
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.614025
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.662940
Train Epoch: 89 - Loss: 0.6301, Accuracy: 94.72%

Test set: Average loss: 0.5193, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.707653
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.658575
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.565253
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.631414
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.635632
Train Epoch: 90 - Loss: 0.6297, Accuracy: 94.78%

Test set: Average loss: 0.5182, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.629295
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.599994
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.602284
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.618079
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.665037
Train Epoch: 91 - Loss: 0.6278, Accuracy: 94.89%

Test set: Average loss: 0.5192, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.603516
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.652842
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.577613
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.614140
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.629649
Train Epoch: 92 - Loss: 0.6276, Accuracy: 94.96%

Test set: Average loss: 0.5178, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.669942
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.699200
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.651773
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.605346
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.638711
Train Epoch: 93 - Loss: 0.6272, Accuracy: 94.87%

Test set: Average loss: 0.5176, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.596995
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.583970
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.604562
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.698965
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.653274
Train Epoch: 94 - Loss: 0.6262, Accuracy: 94.93%

Test set: Average loss: 0.5178, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.627651
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.566891
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.690000
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.652890
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.715047
Train Epoch: 95 - Loss: 0.6249, Accuracy: 95.01%

Test set: Average loss: 0.5173, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.616579
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.628590
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.610898
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.600775
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.579534
Train Epoch: 96 - Loss: 0.6245, Accuracy: 95.04%

Test set: Average loss: 0.5173, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.667655
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.619308
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.619043
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.602139
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.705121
Train Epoch: 97 - Loss: 0.6271, Accuracy: 94.86%

Test set: Average loss: 0.5172, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.678708
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.605401
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.642245
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.610143
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.569815
Train Epoch: 98 - Loss: 0.6242, Accuracy: 95.03%

Test set: Average loss: 0.5174, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.607780
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.648532
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.657147
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.611988
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.676916
Train Epoch: 99 - Loss: 0.6250, Accuracy: 95.02%

Test set: Average loss: 0.5172, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.697605
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.642707
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.618099
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.615782
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.627029
Train Epoch: 100 - Loss: 0.6231, Accuracy: 95.07%

Test set: Average loss: 0.5171, Accuracy: 9957/10000 (99.57%)


Training completed. Best Accuracy: 99.61%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.521590
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.427493
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.061963
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.039317
Train Epoch: 1 [51200/60000 (85%)]	Loss: 0.994203
Train Epoch: 1 - Loss: 1.2908, Accuracy: 71.50%

Test set: Average loss: 0.6712, Accuracy: 9695/10000 (96.95%)

New best model saved with accuracy: 96.95%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.936853
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.830452
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.918471
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.806898
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.869577
Train Epoch: 2 - Loss: 0.8600, Accuracy: 86.63%

Test set: Average loss: 0.5951, Accuracy: 9845/10000 (98.45%)

New best model saved with accuracy: 98.45%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.813814
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.886632
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.826517
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.807425
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.725115
Train Epoch: 3 - Loss: 0.7932, Accuracy: 89.24%

Test set: Average loss: 0.5767, Accuracy: 9868/10000 (98.68%)

New best model saved with accuracy: 98.68%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.765530
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.767494
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.708300
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.791312
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.735468
Train Epoch: 4 - Loss: 0.7587, Accuracy: 90.44%

Test set: Average loss: 0.5653, Accuracy: 9885/10000 (98.85%)

New best model saved with accuracy: 98.85%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.745192
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.819908
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.734586
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.725995
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.723811
Train Epoch: 5 - Loss: 0.7383, Accuracy: 91.18%

Test set: Average loss: 0.5537, Accuracy: 9897/10000 (98.97%)

New best model saved with accuracy: 98.97%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.742564
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.783393
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.776205
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.741202
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.746512
Train Epoch: 6 - Loss: 0.7250, Accuracy: 91.38%

Test set: Average loss: 0.5520, Accuracy: 9902/10000 (99.02%)

New best model saved with accuracy: 99.02%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.761708
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.790721
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.698575
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.715500
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.758982
Train Epoch: 7 - Loss: 0.7118, Accuracy: 92.07%

Test set: Average loss: 0.5488, Accuracy: 9903/10000 (99.03%)

New best model saved with accuracy: 99.03%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.772272
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.717412
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.732826
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.782940
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.740837
Train Epoch: 8 - Loss: 0.7072, Accuracy: 92.21%

Test set: Average loss: 0.5436, Accuracy: 9919/10000 (99.19%)

New best model saved with accuracy: 99.19%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.690330
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.742176
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.685629
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.655615
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.732655
Train Epoch: 9 - Loss: 0.6974, Accuracy: 92.53%

Test set: Average loss: 0.5390, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.685237
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.826076
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.755222
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.691229
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.657421
Train Epoch: 10 - Loss: 0.6920, Accuracy: 92.67%

Test set: Average loss: 0.5384, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.693701
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.695762
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.609835
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.666757
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.715776
Train Epoch: 11 - Loss: 0.6892, Accuracy: 92.73%

Test set: Average loss: 0.5356, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.709256
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.696720
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.711702
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.652191
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.688180
Train Epoch: 12 - Loss: 0.6841, Accuracy: 93.03%

Test set: Average loss: 0.5354, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.640251
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.697777
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.653118
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.650351
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.693355
Train Epoch: 13 - Loss: 0.6807, Accuracy: 93.06%

Test set: Average loss: 0.5319, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.655635
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.666782
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.615407
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.700991
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.647128
Train Epoch: 14 - Loss: 0.6736, Accuracy: 93.29%

Test set: Average loss: 0.5323, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.619179
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.700076
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.705725
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.719344
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.695155
Train Epoch: 15 - Loss: 0.6722, Accuracy: 93.35%

Test set: Average loss: 0.5323, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.684000
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.777791
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.639259
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.650804
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.656863
Train Epoch: 16 - Loss: 0.6699, Accuracy: 93.53%

Test set: Average loss: 0.5308, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.615167
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.638637
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.611438
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.682654
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.707164
Train Epoch: 17 - Loss: 0.6692, Accuracy: 93.57%

Test set: Average loss: 0.5310, Accuracy: 9937/10000 (99.37%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.679183
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.647342
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.647167
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.640236
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.636835
Train Epoch: 18 - Loss: 0.6731, Accuracy: 93.48%

Test set: Average loss: 0.5291, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.649403
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.645961
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.733010
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.677823
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.672390
Train Epoch: 19 - Loss: 0.6698, Accuracy: 93.41%

Test set: Average loss: 0.5296, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.677131
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.636997
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.615976
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.680236
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.751798
Train Epoch: 20 - Loss: 0.6653, Accuracy: 93.63%

Test set: Average loss: 0.5291, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.707762
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.677248
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.683899
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.628971
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.630653
Train Epoch: 21 - Loss: 0.6640, Accuracy: 93.70%

Test set: Average loss: 0.5286, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.618414
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.659481
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.677325
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.709281
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.665962
Train Epoch: 22 - Loss: 0.6623, Accuracy: 93.83%

Test set: Average loss: 0.5278, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.675820
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.717876
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.666108
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.656194
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.691710
Train Epoch: 23 - Loss: 0.6649, Accuracy: 93.55%

Test set: Average loss: 0.5279, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.642748
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.677620
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.652271
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.630142
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.677362
Train Epoch: 24 - Loss: 0.6622, Accuracy: 93.62%

Test set: Average loss: 0.5282, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.665084
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.684320
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.645229
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.651327
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.592285
Train Epoch: 25 - Loss: 0.6609, Accuracy: 93.79%

Test set: Average loss: 0.5281, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.670414
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.719228
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.679111
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.617300
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.668223
Train Epoch: 26 - Loss: 0.6725, Accuracy: 93.36%

Test set: Average loss: 0.5321, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.619249
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.702404
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.641018
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.658381
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.668022
Train Epoch: 27 - Loss: 0.6716, Accuracy: 93.36%

Test set: Average loss: 0.5292, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.637274
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.655959
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.671979
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.660088
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.639926
Train Epoch: 28 - Loss: 0.6702, Accuracy: 93.37%

Test set: Average loss: 0.5285, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.610979
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.694068
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.604926
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.636487
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.649014
Train Epoch: 29 - Loss: 0.6694, Accuracy: 93.34%

Test set: Average loss: 0.5285, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.639953
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.638433
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.698926
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.647718
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.623683
Train Epoch: 30 - Loss: 0.6658, Accuracy: 93.56%

Test set: Average loss: 0.5282, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.654199
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.612595
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.650830
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.688787
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.682979
Train Epoch: 31 - Loss: 0.6625, Accuracy: 93.58%

Test set: Average loss: 0.5277, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.676040
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.659278
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.644838
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.674052
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.681080
Train Epoch: 32 - Loss: 0.6612, Accuracy: 93.68%

Test set: Average loss: 0.5258, Accuracy: 9954/10000 (99.54%)

New best model saved with accuracy: 99.54%
Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.667741
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.670950
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.652903
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.653662
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.636043
Train Epoch: 33 - Loss: 0.6592, Accuracy: 93.74%

Test set: Average loss: 0.5265, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.614029
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.685356
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.648279
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.672928
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.643222
Train Epoch: 34 - Loss: 0.6576, Accuracy: 93.76%

Test set: Average loss: 0.5248, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.616002
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.605685
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.649419
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.591701
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.684008
Train Epoch: 35 - Loss: 0.6545, Accuracy: 94.03%

Test set: Average loss: 0.5232, Accuracy: 9958/10000 (99.58%)

New best model saved with accuracy: 99.58%
Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.609315
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.677184
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.689420
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.666401
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.668179
Train Epoch: 36 - Loss: 0.6556, Accuracy: 93.97%

Test set: Average loss: 0.5239, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.635600
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.688230
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.591559
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.660977
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.652197
Train Epoch: 37 - Loss: 0.6500, Accuracy: 94.07%

Test set: Average loss: 0.5227, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.646509
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.676444
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.604585
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.590449
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.677501
Train Epoch: 38 - Loss: 0.6487, Accuracy: 94.19%

Test set: Average loss: 0.5220, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.599808
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.720339
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.672260
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.608218
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.690719
Train Epoch: 39 - Loss: 0.6478, Accuracy: 94.23%

Test set: Average loss: 0.5221, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.649024
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.620815
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.679040
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.606229
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.666074
Train Epoch: 40 - Loss: 0.6454, Accuracy: 94.38%

Test set: Average loss: 0.5213, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.576246
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.624548
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.613804
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.637071
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.708655
Train Epoch: 41 - Loss: 0.6473, Accuracy: 94.17%

Test set: Average loss: 0.5212, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.607284
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.698977
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.638771
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.721324
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.599835
Train Epoch: 42 - Loss: 0.6442, Accuracy: 94.39%

Test set: Average loss: 0.5206, Accuracy: 9960/10000 (99.60%)

New best model saved with accuracy: 99.60%
Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.587152
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.688686
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.653939
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.761055
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.653092
Train Epoch: 43 - Loss: 0.6431, Accuracy: 94.37%

Test set: Average loss: 0.5205, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.645997
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.607082
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.637894
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.613889
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.616004
Train Epoch: 44 - Loss: 0.6442, Accuracy: 94.30%

Test set: Average loss: 0.5206, Accuracy: 9961/10000 (99.61%)

New best model saved with accuracy: 99.61%
Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.588508
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.664190
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.616935
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.658905
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.602144
Train Epoch: 45 - Loss: 0.6424, Accuracy: 94.39%

Test set: Average loss: 0.5208, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.640860
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.616531
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.578332
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.629058
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.572287
Train Epoch: 46 - Loss: 0.6414, Accuracy: 94.49%

Test set: Average loss: 0.5204, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.591787
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.617143
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.660575
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.574471
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.632715
Train Epoch: 47 - Loss: 0.6394, Accuracy: 94.46%

Test set: Average loss: 0.5200, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.659086
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.633619
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.667588
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.636162
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.632060
Train Epoch: 48 - Loss: 0.6386, Accuracy: 94.54%

Test set: Average loss: 0.5199, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.678063
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.587101
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.600094
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.592815
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.624723
Train Epoch: 49 - Loss: 0.6416, Accuracy: 94.45%

Test set: Average loss: 0.5202, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.747386
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.648816
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.646033
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.649653
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.646772
Train Epoch: 50 - Loss: 0.6387, Accuracy: 94.57%

Test set: Average loss: 0.5200, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.567804
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.613085
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.637689
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.646340
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.599630
Train Epoch: 51 - Loss: 0.6492, Accuracy: 94.09%

Test set: Average loss: 0.5244, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.693166
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.672494
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.654874
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.671692
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.616788
Train Epoch: 52 - Loss: 0.6482, Accuracy: 94.12%

Test set: Average loss: 0.5226, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.720581
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.657203
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.702514
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.638576
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.671570
Train Epoch: 53 - Loss: 0.6510, Accuracy: 93.96%

Test set: Average loss: 0.5232, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.649138
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.626425
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.727332
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.632500
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.594410
Train Epoch: 54 - Loss: 0.6482, Accuracy: 94.11%

Test set: Average loss: 0.5223, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.614405
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.663280
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.629830
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.661642
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.599311
Train Epoch: 55 - Loss: 0.6432, Accuracy: 94.36%

Test set: Average loss: 0.5227, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.678456
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.642927
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.617901
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.646736
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.633842
Train Epoch: 56 - Loss: 0.6452, Accuracy: 94.20%

Test set: Average loss: 0.5204, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.659714
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.643786
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.616227
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.635995
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.662341
Train Epoch: 57 - Loss: 0.6454, Accuracy: 94.22%

Test set: Average loss: 0.5205, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.651290
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.676921
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.595703
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.749551
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.616046
Train Epoch: 58 - Loss: 0.6412, Accuracy: 94.41%

Test set: Average loss: 0.5209, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.622244
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.654975
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.618031
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.616188
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.683093
Train Epoch: 59 - Loss: 0.6438, Accuracy: 94.25%

Test set: Average loss: 0.5194, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.647313
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.614686
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.693295
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.637466
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.654071
Train Epoch: 60 - Loss: 0.6377, Accuracy: 94.54%

Test set: Average loss: 0.5193, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.630459
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.637820
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.597525
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.652225
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.638022
Train Epoch: 61 - Loss: 0.6370, Accuracy: 94.55%

Test set: Average loss: 0.5192, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.645465
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.625291
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.616133
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.673428
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.645628
Train Epoch: 62 - Loss: 0.6390, Accuracy: 94.39%

Test set: Average loss: 0.5187, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.630937
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.617383
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.662370
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.658277
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.583614
Train Epoch: 63 - Loss: 0.6363, Accuracy: 94.53%

Test set: Average loss: 0.5191, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.672775
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.631602
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.551648
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.615750
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.692600
Train Epoch: 64 - Loss: 0.6363, Accuracy: 94.57%

Test set: Average loss: 0.5187, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.612879
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.638497
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.631586
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.607920
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.647261
Train Epoch: 65 - Loss: 0.6361, Accuracy: 94.50%

Test set: Average loss: 0.5180, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.655959
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.607687
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.611899
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.627557
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.588280
Train Epoch: 66 - Loss: 0.6335, Accuracy: 94.74%

Test set: Average loss: 0.5180, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.657636
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.627934
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.623455
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.676730
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.632853
Train Epoch: 67 - Loss: 0.6346, Accuracy: 94.58%

Test set: Average loss: 0.5180, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.678025
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.619903
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.631094
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.658439
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.627030
Train Epoch: 68 - Loss: 0.6320, Accuracy: 94.63%

Test set: Average loss: 0.5175, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.628701
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.655957
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.586455
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.640823
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.623078
Train Epoch: 69 - Loss: 0.6312, Accuracy: 94.73%

Test set: Average loss: 0.5174, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.666637
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.625469
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.563546
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.611892
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.669781
Train Epoch: 70 - Loss: 0.6319, Accuracy: 94.76%

Test set: Average loss: 0.5171, Accuracy: 9962/10000 (99.62%)

New best model saved with accuracy: 99.62%
Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.602201
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.635058
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.656065
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.619001
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.725487
Train Epoch: 71 - Loss: 0.6327, Accuracy: 94.72%

Test set: Average loss: 0.5171, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.603114
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.637045
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.586257
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.590084
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.678147
Train Epoch: 72 - Loss: 0.6304, Accuracy: 94.80%

Test set: Average loss: 0.5169, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.616749
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.618007
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.638262
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.619265
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.620973
Train Epoch: 73 - Loss: 0.6300, Accuracy: 94.80%

Test set: Average loss: 0.5170, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.648394
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.684243
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.601964
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.617411
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.561043
Train Epoch: 74 - Loss: 0.6285, Accuracy: 94.77%

Test set: Average loss: 0.5171, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.657550
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.642413
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.592676
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.634727
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.579727
Train Epoch: 75 - Loss: 0.6285, Accuracy: 94.88%

Test set: Average loss: 0.5170, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.627050
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.595276
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.631543
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.604213
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.668276
Train Epoch: 76 - Loss: 0.6390, Accuracy: 94.25%

Test set: Average loss: 0.5179, Accuracy: 9963/10000 (99.63%)

New best model saved with accuracy: 99.63%
Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.646108
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.616076
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.621236
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.613060
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.700661
Train Epoch: 77 - Loss: 0.6390, Accuracy: 94.35%

Test set: Average loss: 0.5191, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.585632
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.645431
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.695397
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.674456
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.630864
Train Epoch: 78 - Loss: 0.6379, Accuracy: 94.56%

Test set: Average loss: 0.5185, Accuracy: 9965/10000 (99.65%)

New best model saved with accuracy: 99.65%
Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.607268
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.583965
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.578634
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.754377
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.664872
Train Epoch: 79 - Loss: 0.6374, Accuracy: 94.56%

Test set: Average loss: 0.5193, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.729272
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.626880
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.629691
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.646400
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.617357
Train Epoch: 80 - Loss: 0.6365, Accuracy: 94.50%

Test set: Average loss: 0.5179, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.655851
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.631790
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.637647
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.592332
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.681906
Train Epoch: 81 - Loss: 0.6342, Accuracy: 94.50%

Test set: Average loss: 0.5174, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.688779
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.697999
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.603959
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.614895
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.669295
Train Epoch: 82 - Loss: 0.6376, Accuracy: 94.33%

Test set: Average loss: 0.5186, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.635601
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.600424
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.566273
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.624434
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.663375
Train Epoch: 83 - Loss: 0.6358, Accuracy: 94.60%

Test set: Average loss: 0.5169, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.643896
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.624079
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.590472
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.641389
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.648770
Train Epoch: 84 - Loss: 0.6338, Accuracy: 94.57%

Test set: Average loss: 0.5178, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.687644
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.628567
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.638350
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.625707
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.656608
Train Epoch: 85 - Loss: 0.6325, Accuracy: 94.61%

Test set: Average loss: 0.5173, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.625560
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.615390
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.594432
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.626051
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.614879
Train Epoch: 86 - Loss: 0.6300, Accuracy: 94.82%

Test set: Average loss: 0.5174, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.612640
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.673346
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.598815
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.565346
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.623385
Train Epoch: 87 - Loss: 0.6295, Accuracy: 94.67%

Test set: Average loss: 0.5168, Accuracy: 9963/10000 (99.63%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.625565
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.628396
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.619023
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.638823
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.703636
Train Epoch: 88 - Loss: 0.6302, Accuracy: 94.67%

Test set: Average loss: 0.5161, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.589899
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.673361
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.556099
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.663386
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.597420
Train Epoch: 89 - Loss: 0.6279, Accuracy: 94.78%

Test set: Average loss: 0.5169, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.620965
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.584409
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.619092
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.659462
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.625466
Train Epoch: 90 - Loss: 0.6278, Accuracy: 94.75%

Test set: Average loss: 0.5161, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.572177
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.562486
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.594561
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.625997
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.690548
Train Epoch: 91 - Loss: 0.6261, Accuracy: 94.92%

Test set: Average loss: 0.5160, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.625375
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.624884
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.603788
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.663943
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.562048
Train Epoch: 92 - Loss: 0.6259, Accuracy: 94.94%

Test set: Average loss: 0.5157, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.616072
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.615346
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.644400
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.618613
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.641169
Train Epoch: 93 - Loss: 0.6243, Accuracy: 95.08%

Test set: Average loss: 0.5156, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.626161
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.652567
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.587295
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.648289
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.584341
Train Epoch: 94 - Loss: 0.6252, Accuracy: 94.95%

Test set: Average loss: 0.5158, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.572220
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.626851
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.608780
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.642130
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.645666
Train Epoch: 95 - Loss: 0.6231, Accuracy: 95.07%

Test set: Average loss: 0.5159, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.569616
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.640337
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.650783
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.596463
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.597610
Train Epoch: 96 - Loss: 0.6220, Accuracy: 95.13%

Test set: Average loss: 0.5154, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.648600
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.574184
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.662866
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.587770
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.705693
Train Epoch: 97 - Loss: 0.6230, Accuracy: 95.05%

Test set: Average loss: 0.5153, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.591036
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.602300
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.638549
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.608616
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.655280
Train Epoch: 98 - Loss: 0.6236, Accuracy: 95.00%

Test set: Average loss: 0.5155, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.649959
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.653043
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.645334
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.671313
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.622792
Train Epoch: 99 - Loss: 0.6221, Accuracy: 95.14%

Test set: Average loss: 0.5154, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.568459
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.589360
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.637498
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.617588
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.660792
Train Epoch: 100 - Loss: 0.6238, Accuracy: 94.99%

Test set: Average loss: 0.5153, Accuracy: 9961/10000 (99.61%)


Training completed. Best Accuracy: 99.65%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 5.083790
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.322572
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.163211
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.111473
Train Epoch: 1 [51200/60000 (85%)]	Loss: 1.073137
Train Epoch: 1 - Loss: 1.3098, Accuracy: 70.69%

Test set: Average loss: 0.6573, Accuracy: 9705/10000 (97.05%)

New best model saved with accuracy: 97.05%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.927179
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.898539
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.862773
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.894914
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.791909
Train Epoch: 2 - Loss: 0.8656, Accuracy: 86.38%

Test set: Average loss: 0.5919, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.862248
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.769705
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.793316
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.793678
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.749511
Train Epoch: 3 - Loss: 0.7954, Accuracy: 89.02%

Test set: Average loss: 0.5721, Accuracy: 9863/10000 (98.63%)

New best model saved with accuracy: 98.63%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.789096
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.752185
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.768392
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.754220
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.692066
Train Epoch: 4 - Loss: 0.7628, Accuracy: 90.11%

Test set: Average loss: 0.5596, Accuracy: 9889/10000 (98.89%)

New best model saved with accuracy: 98.89%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.732889
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.751469
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.825580
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.719699
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.710704
Train Epoch: 5 - Loss: 0.7356, Accuracy: 91.26%

Test set: Average loss: 0.5575, Accuracy: 9881/10000 (98.81%)

Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.739374
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.803750
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.687337
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.756654
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.715671
Train Epoch: 6 - Loss: 0.7216, Accuracy: 91.64%

Test set: Average loss: 0.5462, Accuracy: 9899/10000 (98.99%)

New best model saved with accuracy: 98.99%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.715193
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.654113
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.651283
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.760484
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.748955
Train Epoch: 7 - Loss: 0.7133, Accuracy: 91.97%

Test set: Average loss: 0.5449, Accuracy: 9918/10000 (99.18%)

New best model saved with accuracy: 99.18%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.683980
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.724766
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.743653
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.747128
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.656684
Train Epoch: 8 - Loss: 0.7064, Accuracy: 92.12%

Test set: Average loss: 0.5398, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.657178
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.694679
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.716839
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.750466
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.700505
Train Epoch: 9 - Loss: 0.6989, Accuracy: 92.41%

Test set: Average loss: 0.5405, Accuracy: 9924/10000 (99.24%)

New best model saved with accuracy: 99.24%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.706196
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.721518
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.670336
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.666737
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.637492
Train Epoch: 10 - Loss: 0.6925, Accuracy: 92.62%

Test set: Average loss: 0.5362, Accuracy: 9922/10000 (99.22%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.699958
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.646504
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.700125
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.704999
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.659452
Train Epoch: 11 - Loss: 0.6867, Accuracy: 92.91%

Test set: Average loss: 0.5366, Accuracy: 9921/10000 (99.21%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.653549
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.713423
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.623553
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.723276
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.700178
Train Epoch: 12 - Loss: 0.6800, Accuracy: 93.05%

Test set: Average loss: 0.5339, Accuracy: 9929/10000 (99.29%)

New best model saved with accuracy: 99.29%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.724605
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.617992
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.694484
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.583619
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.664299
Train Epoch: 13 - Loss: 0.6782, Accuracy: 93.18%

Test set: Average loss: 0.5307, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.673893
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.688385
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.686560
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.639473
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.721316
Train Epoch: 14 - Loss: 0.6760, Accuracy: 93.22%

Test set: Average loss: 0.5305, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.751281
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.674271
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.709203
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.656233
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.719113
Train Epoch: 15 - Loss: 0.6755, Accuracy: 93.25%

Test set: Average loss: 0.5314, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.648016
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.690076
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.694723
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.651270
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.654582
Train Epoch: 16 - Loss: 0.6693, Accuracy: 93.40%

Test set: Average loss: 0.5295, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.713452
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.725994
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.649488
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.667177
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.667437
Train Epoch: 17 - Loss: 0.6664, Accuracy: 93.52%

Test set: Average loss: 0.5304, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.622525
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.651109
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.661531
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.608990
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.684960
Train Epoch: 18 - Loss: 0.6647, Accuracy: 93.68%

Test set: Average loss: 0.5289, Accuracy: 9936/10000 (99.36%)

New best model saved with accuracy: 99.36%
Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.641025
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.669151
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.622049
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.626717
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.675783
Train Epoch: 19 - Loss: 0.6657, Accuracy: 93.66%

Test set: Average loss: 0.5280, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.650791
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.714863
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.622758
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.626709
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.644440
Train Epoch: 20 - Loss: 0.6636, Accuracy: 93.70%

Test set: Average loss: 0.5283, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.627652
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.614649
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.577983
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.650801
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.710236
Train Epoch: 21 - Loss: 0.6641, Accuracy: 93.57%

Test set: Average loss: 0.5280, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.619942
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.673157
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.668379
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.687785
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.678563
Train Epoch: 22 - Loss: 0.6633, Accuracy: 93.61%

Test set: Average loss: 0.5279, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.686950
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.621748
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.660538
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.661979
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.661715
Train Epoch: 23 - Loss: 0.6596, Accuracy: 93.80%

Test set: Average loss: 0.5281, Accuracy: 9938/10000 (99.38%)

New best model saved with accuracy: 99.38%
Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.658255
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.651549
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.665475
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.768784
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.626470
Train Epoch: 24 - Loss: 0.6613, Accuracy: 93.75%

Test set: Average loss: 0.5281, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.654061
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.635326
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.665457
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.678157
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.593640
Train Epoch: 25 - Loss: 0.6585, Accuracy: 93.92%

Test set: Average loss: 0.5278, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.607216
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.610003
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.701782
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.664288
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.626572
Train Epoch: 26 - Loss: 0.6723, Accuracy: 93.34%

Test set: Average loss: 0.5298, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.645854
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.686294
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.663923
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.758137
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.643468
Train Epoch: 27 - Loss: 0.6701, Accuracy: 93.47%

Test set: Average loss: 0.5299, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.701439
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.723823
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.686997
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.594663
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.692582
Train Epoch: 28 - Loss: 0.6686, Accuracy: 93.50%

Test set: Average loss: 0.5274, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.638959
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.639804
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.732432
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.659937
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.684509
Train Epoch: 29 - Loss: 0.6643, Accuracy: 93.46%

Test set: Average loss: 0.5275, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.750871
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.669211
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.695965
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.683276
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.686620
Train Epoch: 30 - Loss: 0.6631, Accuracy: 93.62%

Test set: Average loss: 0.5281, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.700212
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.663555
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.671758
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.617726
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.687613
Train Epoch: 31 - Loss: 0.6633, Accuracy: 93.59%

Test set: Average loss: 0.5259, Accuracy: 9943/10000 (99.43%)

New best model saved with accuracy: 99.43%
Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.662556
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.651003
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.639017
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.672903
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.672504
Train Epoch: 32 - Loss: 0.6598, Accuracy: 93.69%

Test set: Average loss: 0.5242, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.622858
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.670165
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.630378
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.684555
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.630727
Train Epoch: 33 - Loss: 0.6555, Accuracy: 93.79%

Test set: Average loss: 0.5242, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.655614
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.646372
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.712828
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.731897
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.674830
Train Epoch: 34 - Loss: 0.6548, Accuracy: 93.94%

Test set: Average loss: 0.5251, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.681949
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.628682
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.656210
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.639488
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.634676
Train Epoch: 35 - Loss: 0.6509, Accuracy: 93.98%

Test set: Average loss: 0.5244, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.662394
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.617104
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.625557
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.604167
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.614566
Train Epoch: 36 - Loss: 0.6519, Accuracy: 94.03%

Test set: Average loss: 0.5230, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.617531
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.637973
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.656432
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.656456
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.575584
Train Epoch: 37 - Loss: 0.6481, Accuracy: 94.15%

Test set: Average loss: 0.5226, Accuracy: 9951/10000 (99.51%)

New best model saved with accuracy: 99.51%
Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.643555
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.609833
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.579067
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.672479
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.658493
Train Epoch: 38 - Loss: 0.6493, Accuracy: 94.10%

Test set: Average loss: 0.5222, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.633940
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.633680
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.621758
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.614572
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.642128
Train Epoch: 39 - Loss: 0.6451, Accuracy: 94.25%

Test set: Average loss: 0.5216, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.644999
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.674833
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.637333
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.689130
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.587426
Train Epoch: 40 - Loss: 0.6449, Accuracy: 94.28%

Test set: Average loss: 0.5208, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.618858
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.647325
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.644723
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.612918
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.618723
Train Epoch: 41 - Loss: 0.6422, Accuracy: 94.41%

Test set: Average loss: 0.5210, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.659276
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.666404
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.689335
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.623821
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.630395
Train Epoch: 42 - Loss: 0.6389, Accuracy: 94.57%

Test set: Average loss: 0.5211, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.578370
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.739256
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.669479
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.598950
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.673063
Train Epoch: 43 - Loss: 0.6384, Accuracy: 94.53%

Test set: Average loss: 0.5205, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.640271
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.645494
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.690248
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.582777
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.650351
Train Epoch: 44 - Loss: 0.6400, Accuracy: 94.38%

Test set: Average loss: 0.5203, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.637005
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.621034
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.665743
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.660931
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.617785
Train Epoch: 45 - Loss: 0.6391, Accuracy: 94.41%

Test set: Average loss: 0.5201, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.689052
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.672865
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.705836
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.669806
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.579806
Train Epoch: 46 - Loss: 0.6404, Accuracy: 94.39%

Test set: Average loss: 0.5200, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.637706
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.611457
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.656625
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.607909
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.593997
Train Epoch: 47 - Loss: 0.6383, Accuracy: 94.65%

Test set: Average loss: 0.5201, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.614589
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.727238
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.616171
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.629273
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.707962
Train Epoch: 48 - Loss: 0.6371, Accuracy: 94.64%

Test set: Average loss: 0.5203, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.648933
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.619143
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.596291
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.650160
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.716923
Train Epoch: 49 - Loss: 0.6360, Accuracy: 94.59%

Test set: Average loss: 0.5199, Accuracy: 9954/10000 (99.54%)

New best model saved with accuracy: 99.54%
Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.598292
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.612999
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.594740
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.597924
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.581216
Train Epoch: 50 - Loss: 0.6354, Accuracy: 94.67%

Test set: Average loss: 0.5197, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.560433
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.656203
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.628906
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.604574
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.668858
Train Epoch: 51 - Loss: 0.6489, Accuracy: 94.00%

Test set: Average loss: 0.5240, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.591322
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.609778
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.682191
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.623401
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.616788
Train Epoch: 52 - Loss: 0.6486, Accuracy: 94.07%

Test set: Average loss: 0.5223, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.603795
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.650599
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.628295
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.613404
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.686634
Train Epoch: 53 - Loss: 0.6461, Accuracy: 94.21%

Test set: Average loss: 0.5222, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.647767
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.606051
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.678444
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.673876
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.661447
Train Epoch: 54 - Loss: 0.6462, Accuracy: 94.20%

Test set: Average loss: 0.5210, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.631596
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.607814
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.646454
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.620713
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.594144
Train Epoch: 55 - Loss: 0.6437, Accuracy: 94.32%

Test set: Average loss: 0.5199, Accuracy: 9955/10000 (99.55%)

New best model saved with accuracy: 99.55%
Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.616297
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.676072
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.575147
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.670960
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.646945
Train Epoch: 56 - Loss: 0.6428, Accuracy: 94.26%

Test set: Average loss: 0.5207, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.593721
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.657799
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.670951
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.584735
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.663350
Train Epoch: 57 - Loss: 0.6405, Accuracy: 94.48%

Test set: Average loss: 0.5195, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.614901
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.648394
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.644996
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.639626
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.665317
Train Epoch: 58 - Loss: 0.6403, Accuracy: 94.36%

Test set: Average loss: 0.5201, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.636614
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.634040
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.673717
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.693393
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.626386
Train Epoch: 59 - Loss: 0.6396, Accuracy: 94.39%

Test set: Average loss: 0.5201, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.692927
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.668774
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.630140
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.613579
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.654348
Train Epoch: 60 - Loss: 0.6361, Accuracy: 94.62%

Test set: Average loss: 0.5187, Accuracy: 9959/10000 (99.59%)

New best model saved with accuracy: 99.59%
Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.587741
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.637876
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.728523
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.638960
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.647747
Train Epoch: 61 - Loss: 0.6357, Accuracy: 94.54%

Test set: Average loss: 0.5190, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.644355
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.634648
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.728196
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.676976
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.589482
Train Epoch: 62 - Loss: 0.6356, Accuracy: 94.54%

Test set: Average loss: 0.5180, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.640662
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.623098
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.655858
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.672436
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.625978
Train Epoch: 63 - Loss: 0.6360, Accuracy: 94.50%

Test set: Average loss: 0.5178, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.642402
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.609985
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.575619
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.648700
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.629538
Train Epoch: 64 - Loss: 0.6343, Accuracy: 94.58%

Test set: Average loss: 0.5177, Accuracy: 9964/10000 (99.64%)

New best model saved with accuracy: 99.64%
Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.614834
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.781001
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.656043
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.620083
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.617427
Train Epoch: 65 - Loss: 0.6303, Accuracy: 94.78%

Test set: Average loss: 0.5184, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.696283
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.648728
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.654198
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.573906
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.687955
Train Epoch: 66 - Loss: 0.6302, Accuracy: 94.69%

Test set: Average loss: 0.5174, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.588662
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.580116
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.583497
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.561205
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.626644
Train Epoch: 67 - Loss: 0.6313, Accuracy: 94.64%

Test set: Average loss: 0.5172, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.605137
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.588615
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.622513
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.564901
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.615667
Train Epoch: 68 - Loss: 0.6289, Accuracy: 94.75%

Test set: Average loss: 0.5176, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.625079
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.593814
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.579196
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.634342
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.649939
Train Epoch: 69 - Loss: 0.6286, Accuracy: 94.76%

Test set: Average loss: 0.5172, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.622374
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.619382
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.635684
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.645220
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.651564
Train Epoch: 70 - Loss: 0.6275, Accuracy: 94.88%

Test set: Average loss: 0.5171, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.613126
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.668402
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.612505
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.623774
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.647065
Train Epoch: 71 - Loss: 0.6302, Accuracy: 94.79%

Test set: Average loss: 0.5171, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.651623
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.576989
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.622493
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.612476
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.610370
Train Epoch: 72 - Loss: 0.6271, Accuracy: 94.95%

Test set: Average loss: 0.5165, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.698382
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.617543
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.595542
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.698802
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.612014
Train Epoch: 73 - Loss: 0.6269, Accuracy: 94.86%

Test set: Average loss: 0.5166, Accuracy: 9963/10000 (99.63%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.633487
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.628874
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.618689
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.616186
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.627997
Train Epoch: 74 - Loss: 0.6275, Accuracy: 94.90%

Test set: Average loss: 0.5166, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.559027
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.709001
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.607335
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.660760
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.610759
Train Epoch: 75 - Loss: 0.6257, Accuracy: 94.94%

Test set: Average loss: 0.5165, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.644481
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.608191
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.584283
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.634596
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.602704
Train Epoch: 76 - Loss: 0.6389, Accuracy: 94.43%

Test set: Average loss: 0.5183, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.662253
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.639687
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.599646
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.588641
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.618918
Train Epoch: 77 - Loss: 0.6370, Accuracy: 94.39%

Test set: Average loss: 0.5187, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.678839
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.584846
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.632861
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.610794
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.605826
Train Epoch: 78 - Loss: 0.6356, Accuracy: 94.54%

Test set: Average loss: 0.5182, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.695125
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.641488
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.621818
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.602171
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.610924
Train Epoch: 79 - Loss: 0.6347, Accuracy: 94.51%

Test set: Average loss: 0.5173, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.624483
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.600078
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.714224
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.662010
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.641138
Train Epoch: 80 - Loss: 0.6362, Accuracy: 94.47%

Test set: Average loss: 0.5175, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.630400
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.629957
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.604282
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.616657
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.641251
Train Epoch: 81 - Loss: 0.6349, Accuracy: 94.45%

Test set: Average loss: 0.5182, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.649183
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.576437
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.659755
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.609658
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.696479
Train Epoch: 82 - Loss: 0.6313, Accuracy: 94.68%

Test set: Average loss: 0.5172, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.595194
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.631948
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.639862
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.675131
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.648144
Train Epoch: 83 - Loss: 0.6316, Accuracy: 94.71%

Test set: Average loss: 0.5176, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.672810
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.624372
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.590748
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.640113
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.613540
Train Epoch: 84 - Loss: 0.6302, Accuracy: 94.77%

Test set: Average loss: 0.5177, Accuracy: 9963/10000 (99.63%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.621137
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.642393
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.635952
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.565468
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.567254
Train Epoch: 85 - Loss: 0.6303, Accuracy: 94.75%

Test set: Average loss: 0.5180, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.593554
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.586642
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.631181
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.589780
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.647550
Train Epoch: 86 - Loss: 0.6285, Accuracy: 94.77%

Test set: Average loss: 0.5164, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.627049
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.596594
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.646463
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.605247
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.588833
Train Epoch: 87 - Loss: 0.6293, Accuracy: 94.79%

Test set: Average loss: 0.5158, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.692076
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.586077
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.568435
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.644743
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.631819
Train Epoch: 88 - Loss: 0.6242, Accuracy: 94.94%

Test set: Average loss: 0.5162, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.613515
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.641796
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.667015
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.621401
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.576139
Train Epoch: 89 - Loss: 0.6258, Accuracy: 94.89%

Test set: Average loss: 0.5159, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.596966
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.586159
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.562017
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.634643
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.671056
Train Epoch: 90 - Loss: 0.6246, Accuracy: 94.89%

Test set: Average loss: 0.5157, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.603503
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.624445
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.632970
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.607403
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.642031
Train Epoch: 91 - Loss: 0.6246, Accuracy: 94.95%

Test set: Average loss: 0.5164, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.584790
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.581031
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.659056
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.635190
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.552984
Train Epoch: 92 - Loss: 0.6232, Accuracy: 95.01%

Test set: Average loss: 0.5157, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.627854
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.630700
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.688939
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.594077
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.684501
Train Epoch: 93 - Loss: 0.6237, Accuracy: 94.97%

Test set: Average loss: 0.5152, Accuracy: 9966/10000 (99.66%)

New best model saved with accuracy: 99.66%
Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.618653
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.677911
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.578150
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.653169
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.566673
Train Epoch: 94 - Loss: 0.6233, Accuracy: 94.97%

Test set: Average loss: 0.5154, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.581633
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.626884
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.642166
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.625618
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.594382
Train Epoch: 95 - Loss: 0.6195, Accuracy: 95.17%

Test set: Average loss: 0.5153, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.593846
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.652861
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.607344
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.590140
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.604271
Train Epoch: 96 - Loss: 0.6213, Accuracy: 95.03%

Test set: Average loss: 0.5151, Accuracy: 9964/10000 (99.64%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.675450
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.585925
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.641004
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.641959
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.575652
Train Epoch: 97 - Loss: 0.6192, Accuracy: 95.16%

Test set: Average loss: 0.5153, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.578116
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.625354
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.657468
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.616344
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.612882
Train Epoch: 98 - Loss: 0.6227, Accuracy: 95.00%

Test set: Average loss: 0.5150, Accuracy: 9965/10000 (99.65%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.566176
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.650335
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.566264
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.591656
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.636211
Train Epoch: 99 - Loss: 0.6190, Accuracy: 95.15%

Test set: Average loss: 0.5150, Accuracy: 9965/10000 (99.65%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.596962
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.623708
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.602850
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.629510
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.684207
Train Epoch: 100 - Loss: 0.6197, Accuracy: 95.14%

Test set: Average loss: 0.5150, Accuracy: 9966/10000 (99.66%)


Training completed. Best Accuracy: 99.66%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.840779
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.517509
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.169856
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.000081
Train Epoch: 1 [51200/60000 (85%)]	Loss: 1.006178
Train Epoch: 1 - Loss: 1.2939, Accuracy: 70.62%

Test set: Average loss: 0.6536, Accuracy: 9691/10000 (96.91%)

New best model saved with accuracy: 96.91%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.872014
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.804507
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.891124
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.852644
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.891662
Train Epoch: 2 - Loss: 0.8608, Accuracy: 86.42%

Test set: Average loss: 0.6046, Accuracy: 9763/10000 (97.63%)

New best model saved with accuracy: 97.63%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.746078
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.809765
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.834193
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.841866
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.779559
Train Epoch: 3 - Loss: 0.7992, Accuracy: 88.69%

Test set: Average loss: 0.5760, Accuracy: 9838/10000 (98.38%)

New best model saved with accuracy: 98.38%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.794758
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.814098
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.738903
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.734580
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.719938
Train Epoch: 4 - Loss: 0.7637, Accuracy: 90.01%

Test set: Average loss: 0.5604, Accuracy: 9868/10000 (98.68%)

New best model saved with accuracy: 98.68%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.737319
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.767083
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.731909
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.681702
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.747325
Train Epoch: 5 - Loss: 0.7438, Accuracy: 90.77%

Test set: Average loss: 0.5555, Accuracy: 9896/10000 (98.96%)

New best model saved with accuracy: 98.96%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.700566
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.692535
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.698222
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.751437
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.678192
Train Epoch: 6 - Loss: 0.7277, Accuracy: 91.22%

Test set: Average loss: 0.5479, Accuracy: 9909/10000 (99.09%)

New best model saved with accuracy: 99.09%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.670276
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.700486
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.694304
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.699996
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.720986
Train Epoch: 7 - Loss: 0.7187, Accuracy: 91.67%

Test set: Average loss: 0.5465, Accuracy: 9911/10000 (99.11%)

New best model saved with accuracy: 99.11%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.674277
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.784388
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.725711
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.680477
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.727889
Train Epoch: 8 - Loss: 0.7077, Accuracy: 91.99%

Test set: Average loss: 0.5393, Accuracy: 9921/10000 (99.21%)

New best model saved with accuracy: 99.21%
Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.705997
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.674504
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.684745
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.656876
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.657134
Train Epoch: 9 - Loss: 0.7048, Accuracy: 91.94%

Test set: Average loss: 0.5389, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.692756
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.721212
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.702416
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.609135
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.736980
Train Epoch: 10 - Loss: 0.6940, Accuracy: 92.39%

Test set: Average loss: 0.5372, Accuracy: 9931/10000 (99.31%)

New best model saved with accuracy: 99.31%
Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.663554
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.681048
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.657730
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.649158
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.727933
Train Epoch: 11 - Loss: 0.6886, Accuracy: 92.78%

Test set: Average loss: 0.5370, Accuracy: 9930/10000 (99.30%)

Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.665617
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.627139
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.764282
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.649623
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.759541
Train Epoch: 12 - Loss: 0.6829, Accuracy: 92.92%

Test set: Average loss: 0.5340, Accuracy: 9940/10000 (99.40%)

New best model saved with accuracy: 99.40%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.691517
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.619899
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.680018
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.685016
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.656898
Train Epoch: 13 - Loss: 0.6825, Accuracy: 93.02%

Test set: Average loss: 0.5313, Accuracy: 9935/10000 (99.35%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.642405
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.741429
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.670833
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.755794
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.678412
Train Epoch: 14 - Loss: 0.6789, Accuracy: 93.08%

Test set: Average loss: 0.5299, Accuracy: 9942/10000 (99.42%)

New best model saved with accuracy: 99.42%
Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.703473
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.620466
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.679580
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.662633
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.675559
Train Epoch: 15 - Loss: 0.6747, Accuracy: 93.10%

Test set: Average loss: 0.5291, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.633085
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.648128
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.675789
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.669074
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.624247
Train Epoch: 16 - Loss: 0.6709, Accuracy: 93.37%

Test set: Average loss: 0.5283, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.717224
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.676607
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.665855
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.640323
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.664784
Train Epoch: 17 - Loss: 0.6729, Accuracy: 93.24%

Test set: Average loss: 0.5285, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.662550
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.619238
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.676252
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.634147
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.633041
Train Epoch: 18 - Loss: 0.6653, Accuracy: 93.52%

Test set: Average loss: 0.5277, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.662801
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.649893
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.663674
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.690801
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.623916
Train Epoch: 19 - Loss: 0.6669, Accuracy: 93.54%

Test set: Average loss: 0.5273, Accuracy: 9949/10000 (99.49%)

New best model saved with accuracy: 99.49%
Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.620679
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.656487
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.603129
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.693243
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.670616
Train Epoch: 20 - Loss: 0.6644, Accuracy: 93.55%

Test set: Average loss: 0.5266, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.691370
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.637666
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.656201
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.723063
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.710882
Train Epoch: 21 - Loss: 0.6634, Accuracy: 93.58%

Test set: Average loss: 0.5269, Accuracy: 9950/10000 (99.50%)

New best model saved with accuracy: 99.50%
Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.633103
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.689783
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.667503
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.658948
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.655751
Train Epoch: 22 - Loss: 0.6637, Accuracy: 93.63%

Test set: Average loss: 0.5270, Accuracy: 9951/10000 (99.51%)

New best model saved with accuracy: 99.51%
Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.630304
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.660467
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.703645
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.621720
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.677223
Train Epoch: 23 - Loss: 0.6628, Accuracy: 93.65%

Test set: Average loss: 0.5264, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.631187
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.587824
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.711752
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.678163
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.677803
Train Epoch: 24 - Loss: 0.6641, Accuracy: 93.68%

Test set: Average loss: 0.5262, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.715499
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.668405
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.666440
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.627181
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.708109
Train Epoch: 25 - Loss: 0.6610, Accuracy: 93.74%

Test set: Average loss: 0.5265, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.704481
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.708535
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.685960
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.684900
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.665598
Train Epoch: 26 - Loss: 0.6762, Accuracy: 92.89%

Test set: Average loss: 0.5300, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.665781
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.624604
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.642163
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.798830
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.692702
Train Epoch: 27 - Loss: 0.6731, Accuracy: 93.21%

Test set: Average loss: 0.5302, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.586932
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.659126
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.739585
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.648664
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.744789
Train Epoch: 28 - Loss: 0.6712, Accuracy: 93.12%

Test set: Average loss: 0.5266, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.693847
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.681254
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.721327
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.701324
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.667557
Train Epoch: 29 - Loss: 0.6702, Accuracy: 93.20%

Test set: Average loss: 0.5265, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.749631
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.603566
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.672577
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.637315
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.681252
Train Epoch: 30 - Loss: 0.6659, Accuracy: 93.44%

Test set: Average loss: 0.5273, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.716597
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.672791
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.619938
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.647728
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.735981
Train Epoch: 31 - Loss: 0.6642, Accuracy: 93.38%

Test set: Average loss: 0.5264, Accuracy: 9941/10000 (99.41%)

Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.682218
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.639627
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.701487
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.676428
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.603563
Train Epoch: 32 - Loss: 0.6593, Accuracy: 93.69%

Test set: Average loss: 0.5256, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.703200
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.653083
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.688572
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.622583
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.712148
Train Epoch: 33 - Loss: 0.6571, Accuracy: 93.77%

Test set: Average loss: 0.5263, Accuracy: 9936/10000 (99.36%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.678203
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.748536
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.670712
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.696547
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.696265
Train Epoch: 34 - Loss: 0.6552, Accuracy: 93.85%

Test set: Average loss: 0.5255, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.658168
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.632283
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.587888
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.600799
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.627753
Train Epoch: 35 - Loss: 0.6541, Accuracy: 93.80%

Test set: Average loss: 0.5242, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.652429
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.626445
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.657216
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.663360
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.724890
Train Epoch: 36 - Loss: 0.6527, Accuracy: 93.88%

Test set: Average loss: 0.5228, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.638002
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.623812
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.682366
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.671748
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.708269
Train Epoch: 37 - Loss: 0.6484, Accuracy: 94.12%

Test set: Average loss: 0.5215, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.669229
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.669458
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.706521
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.617315
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.630359
Train Epoch: 38 - Loss: 0.6498, Accuracy: 93.94%

Test set: Average loss: 0.5216, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.600900
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.666601
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.654683
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.631076
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.619496
Train Epoch: 39 - Loss: 0.6482, Accuracy: 94.14%

Test set: Average loss: 0.5219, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.607308
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.656993
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.637931
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.690822
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.617569
Train Epoch: 40 - Loss: 0.6486, Accuracy: 94.08%

Test set: Average loss: 0.5224, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.634567
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.624733
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.643902
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.699696
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.652681
Train Epoch: 41 - Loss: 0.6439, Accuracy: 94.20%

Test set: Average loss: 0.5218, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.690350
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.618572
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.622110
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.682046
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.638659
Train Epoch: 42 - Loss: 0.6437, Accuracy: 94.25%

Test set: Average loss: 0.5206, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.603528
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.662367
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.671071
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.695124
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.665450
Train Epoch: 43 - Loss: 0.6410, Accuracy: 94.31%

Test set: Average loss: 0.5205, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.667093
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.616016
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.605045
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.703471
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.711476
Train Epoch: 44 - Loss: 0.6395, Accuracy: 94.45%

Test set: Average loss: 0.5209, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.645067
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.663401
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.615921
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.664920
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.613900
Train Epoch: 45 - Loss: 0.6370, Accuracy: 94.65%

Test set: Average loss: 0.5198, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.622704
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.563811
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.664820
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.635491
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.638689
Train Epoch: 46 - Loss: 0.6384, Accuracy: 94.55%

Test set: Average loss: 0.5204, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.601589
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.633301
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.588796
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.663613
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.644860
Train Epoch: 47 - Loss: 0.6392, Accuracy: 94.49%

Test set: Average loss: 0.5197, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.653590
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.643910
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.692044
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.617461
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.625446
Train Epoch: 48 - Loss: 0.6364, Accuracy: 94.51%

Test set: Average loss: 0.5201, Accuracy: 9954/10000 (99.54%)

New best model saved with accuracy: 99.54%
Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.581537
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.683988
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.633556
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.636195
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.642513
Train Epoch: 49 - Loss: 0.6369, Accuracy: 94.54%

Test set: Average loss: 0.5202, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.719974
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.612457
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.595910
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.642886
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.644692
Train Epoch: 50 - Loss: 0.6365, Accuracy: 94.59%

Test set: Average loss: 0.5204, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.632848
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.610635
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.620868
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.618458
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.737555
Train Epoch: 51 - Loss: 0.6492, Accuracy: 93.96%

Test set: Average loss: 0.5214, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.703283
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.643560
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.575064
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.614599
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.646584
Train Epoch: 52 - Loss: 0.6470, Accuracy: 94.03%

Test set: Average loss: 0.5230, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.680598
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.652960
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.677050
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.732468
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.666010
Train Epoch: 53 - Loss: 0.6491, Accuracy: 93.98%

Test set: Average loss: 0.5225, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.616073
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.694395
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.621315
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.605226
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.677686
Train Epoch: 54 - Loss: 0.6459, Accuracy: 94.19%

Test set: Average loss: 0.5210, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.726842
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.672932
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.652080
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.619089
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.643415
Train Epoch: 55 - Loss: 0.6453, Accuracy: 94.14%

Test set: Average loss: 0.5232, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.633481
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.586171
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.659443
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.643168
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.651728
Train Epoch: 56 - Loss: 0.6452, Accuracy: 94.18%

Test set: Average loss: 0.5196, Accuracy: 9956/10000 (99.56%)

New best model saved with accuracy: 99.56%
Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.619236
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.687436
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.697052
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.721725
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.674947
Train Epoch: 57 - Loss: 0.6451, Accuracy: 94.17%

Test set: Average loss: 0.5202, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.644389
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.612589
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.634491
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.652850
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.625846
Train Epoch: 58 - Loss: 0.6418, Accuracy: 94.27%

Test set: Average loss: 0.5199, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.670984
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.630356
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.665946
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.695331
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.619269
Train Epoch: 59 - Loss: 0.6406, Accuracy: 94.25%

Test set: Average loss: 0.5207, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.574679
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.611805
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.660067
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.622453
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.601854
Train Epoch: 60 - Loss: 0.6376, Accuracy: 94.49%

Test set: Average loss: 0.5199, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.650888
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.615821
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.631449
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.629964
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.628189
Train Epoch: 61 - Loss: 0.6401, Accuracy: 94.28%

Test set: Average loss: 0.5198, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.628827
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.617348
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.657207
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.650561
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.734192
Train Epoch: 62 - Loss: 0.6377, Accuracy: 94.42%

Test set: Average loss: 0.5188, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.542387
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.636635
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.610460
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.596822
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.563243
Train Epoch: 63 - Loss: 0.6356, Accuracy: 94.57%

Test set: Average loss: 0.5190, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.633149
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.655183
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.668553
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.672418
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.599308
Train Epoch: 64 - Loss: 0.6357, Accuracy: 94.54%

Test set: Average loss: 0.5182, Accuracy: 9958/10000 (99.58%)

New best model saved with accuracy: 99.58%
Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.634024
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.657552
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.675117
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.624184
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.660638
Train Epoch: 65 - Loss: 0.6338, Accuracy: 94.55%

Test set: Average loss: 0.5187, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.641256
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.715929
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.680860
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.672750
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.604739
Train Epoch: 66 - Loss: 0.6329, Accuracy: 94.67%

Test set: Average loss: 0.5184, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.608852
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.589224
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.574268
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.627341
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.616252
Train Epoch: 67 - Loss: 0.6318, Accuracy: 94.69%

Test set: Average loss: 0.5181, Accuracy: 9960/10000 (99.60%)

New best model saved with accuracy: 99.60%
Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.633813
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.642763
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.661453
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.618059
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.623513
Train Epoch: 68 - Loss: 0.6338, Accuracy: 94.62%

Test set: Average loss: 0.5188, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.614083
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.682984
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.644118
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.639249
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.642435
Train Epoch: 69 - Loss: 0.6285, Accuracy: 94.75%

Test set: Average loss: 0.5186, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.661028
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.598924
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.615506
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.669200
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.606272
Train Epoch: 70 - Loss: 0.6291, Accuracy: 94.78%

Test set: Average loss: 0.5178, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.569267
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.580459
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.610342
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.671239
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.605590
Train Epoch: 71 - Loss: 0.6275, Accuracy: 94.89%

Test set: Average loss: 0.5173, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.614920
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.642199
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.621276
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.641965
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.639247
Train Epoch: 72 - Loss: 0.6283, Accuracy: 94.84%

Test set: Average loss: 0.5175, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.623900
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.694758
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.672364
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.619266
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.666159
Train Epoch: 73 - Loss: 0.6270, Accuracy: 94.88%

Test set: Average loss: 0.5176, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.674474
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.593018
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.614602
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.557618
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.649605
Train Epoch: 74 - Loss: 0.6256, Accuracy: 94.89%

Test set: Average loss: 0.5176, Accuracy: 9955/10000 (99.55%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.631557
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.590956
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.607644
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.660697
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.640438
Train Epoch: 75 - Loss: 0.6278, Accuracy: 94.86%

Test set: Average loss: 0.5176, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.728150
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.603366
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.632905
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.666963
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.666849
Train Epoch: 76 - Loss: 0.6404, Accuracy: 94.25%

Test set: Average loss: 0.5183, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.599904
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.582598
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.614557
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.615645
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.669605
Train Epoch: 77 - Loss: 0.6385, Accuracy: 94.44%

Test set: Average loss: 0.5195, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.616469
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.590459
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.651586
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.715022
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.625838
Train Epoch: 78 - Loss: 0.6352, Accuracy: 94.54%

Test set: Average loss: 0.5202, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.688013
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.636100
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.615905
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.641526
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.682202
Train Epoch: 79 - Loss: 0.6386, Accuracy: 94.31%

Test set: Average loss: 0.5191, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.620379
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.639411
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.574835
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.665014
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.659715
Train Epoch: 80 - Loss: 0.6364, Accuracy: 94.46%

Test set: Average loss: 0.5183, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.598836
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.640548
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.609939
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.616236
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.609258
Train Epoch: 81 - Loss: 0.6351, Accuracy: 94.47%

Test set: Average loss: 0.5179, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.603482
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.611506
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.650882
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.666749
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.629377
Train Epoch: 82 - Loss: 0.6355, Accuracy: 94.45%

Test set: Average loss: 0.5202, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.668615
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.614536
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.604013
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.611828
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.602684
Train Epoch: 83 - Loss: 0.6328, Accuracy: 94.58%

Test set: Average loss: 0.5187, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.622468
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.645693
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.621894
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.561977
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.594071
Train Epoch: 84 - Loss: 0.6334, Accuracy: 94.58%

Test set: Average loss: 0.5179, Accuracy: 9959/10000 (99.59%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.702484
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.638225
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.632444
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.628795
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.667076
Train Epoch: 85 - Loss: 0.6298, Accuracy: 94.73%

Test set: Average loss: 0.5176, Accuracy: 9958/10000 (99.58%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.666089
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.578881
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.721783
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.619900
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.609835
Train Epoch: 86 - Loss: 0.6302, Accuracy: 94.71%

Test set: Average loss: 0.5172, Accuracy: 9957/10000 (99.57%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.629451
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.650783
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.656657
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.669979
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.616779
Train Epoch: 87 - Loss: 0.6313, Accuracy: 94.58%

Test set: Average loss: 0.5174, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.609646
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.630716
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.659428
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.623404
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.651890
Train Epoch: 88 - Loss: 0.6278, Accuracy: 94.86%

Test set: Average loss: 0.5168, Accuracy: 9956/10000 (99.56%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.613402
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.578796
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.657803
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.696012
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.597961
Train Epoch: 89 - Loss: 0.6250, Accuracy: 94.96%

Test set: Average loss: 0.5161, Accuracy: 9961/10000 (99.61%)

New best model saved with accuracy: 99.61%
Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.615095
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.663221
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.679875
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.616318
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.648577
Train Epoch: 90 - Loss: 0.6270, Accuracy: 94.83%

Test set: Average loss: 0.5162, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.663308
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.602690
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.595833
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.634408
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.628612
Train Epoch: 91 - Loss: 0.6256, Accuracy: 94.83%

Test set: Average loss: 0.5160, Accuracy: 9963/10000 (99.63%)

New best model saved with accuracy: 99.63%
Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.610160
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.587756
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.645578
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.705064
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.609225
Train Epoch: 92 - Loss: 0.6221, Accuracy: 94.99%

Test set: Average loss: 0.5163, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.599391
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.621053
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.617293
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.636027
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.620313
Train Epoch: 93 - Loss: 0.6221, Accuracy: 95.07%

Test set: Average loss: 0.5158, Accuracy: 9963/10000 (99.63%)

Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.580961
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.657985
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.588727
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.601905
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.736572
Train Epoch: 94 - Loss: 0.6253, Accuracy: 94.89%

Test set: Average loss: 0.5158, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.615272
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.655025
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.643911
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.677524
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.602035
Train Epoch: 95 - Loss: 0.6239, Accuracy: 94.98%

Test set: Average loss: 0.5159, Accuracy: 9961/10000 (99.61%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.616004
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.585616
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.636012
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.646748
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.640926
Train Epoch: 96 - Loss: 0.6242, Accuracy: 94.89%

Test set: Average loss: 0.5155, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.609167
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.649239
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.634150
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.579971
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.636211
Train Epoch: 97 - Loss: 0.6219, Accuracy: 94.99%

Test set: Average loss: 0.5155, Accuracy: 9964/10000 (99.64%)

New best model saved with accuracy: 99.64%
Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.634437
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.578629
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.609167
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.605517
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.652865
Train Epoch: 98 - Loss: 0.6233, Accuracy: 95.04%

Test set: Average loss: 0.5155, Accuracy: 9960/10000 (99.60%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.652665
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.612514
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.625185
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.635734
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.635982
Train Epoch: 99 - Loss: 0.6219, Accuracy: 95.08%

Test set: Average loss: 0.5156, Accuracy: 9962/10000 (99.62%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.569185
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.587895
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.632971
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.617620
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.681848
Train Epoch: 100 - Loss: 0.6226, Accuracy: 94.98%

Test set: Average loss: 0.5155, Accuracy: 9961/10000 (99.61%)


Training completed. Best Accuracy: 99.64%
Current learning rate: 0.001000
Train Epoch: 1 [0/60000 (0%)]	Loss: 4.841670
Train Epoch: 1 [12800/60000 (21%)]	Loss: 1.415318
Train Epoch: 1 [25600/60000 (43%)]	Loss: 1.154640
Train Epoch: 1 [38400/60000 (64%)]	Loss: 1.019594
Train Epoch: 1 [51200/60000 (85%)]	Loss: 1.031589
Train Epoch: 1 - Loss: 1.3077, Accuracy: 70.22%

Test set: Average loss: 0.6656, Accuracy: 9677/10000 (96.77%)

New best model saved with accuracy: 96.77%
Current learning rate: 0.000996
Train Epoch: 2 [0/60000 (0%)]	Loss: 0.942861
Train Epoch: 2 [12800/60000 (21%)]	Loss: 0.843390
Train Epoch: 2 [25600/60000 (43%)]	Loss: 0.952938
Train Epoch: 2 [38400/60000 (64%)]	Loss: 0.842662
Train Epoch: 2 [51200/60000 (85%)]	Loss: 0.898639
Train Epoch: 2 - Loss: 0.8852, Accuracy: 85.23%

Test set: Average loss: 0.6096, Accuracy: 9787/10000 (97.87%)

New best model saved with accuracy: 97.87%
Current learning rate: 0.000984
Train Epoch: 3 [0/60000 (0%)]	Loss: 0.782741
Train Epoch: 3 [12800/60000 (21%)]	Loss: 0.938421
Train Epoch: 3 [25600/60000 (43%)]	Loss: 0.794269
Train Epoch: 3 [38400/60000 (64%)]	Loss: 0.838723
Train Epoch: 3 [51200/60000 (85%)]	Loss: 0.770979
Train Epoch: 3 - Loss: 0.8151, Accuracy: 87.97%

Test set: Average loss: 0.5837, Accuracy: 9841/10000 (98.41%)

New best model saved with accuracy: 98.41%
Current learning rate: 0.000965
Train Epoch: 4 [0/60000 (0%)]	Loss: 0.693329
Train Epoch: 4 [12800/60000 (21%)]	Loss: 0.792340
Train Epoch: 4 [25600/60000 (43%)]	Loss: 0.781694
Train Epoch: 4 [38400/60000 (64%)]	Loss: 0.793418
Train Epoch: 4 [51200/60000 (85%)]	Loss: 0.712831
Train Epoch: 4 - Loss: 0.7762, Accuracy: 89.49%

Test set: Average loss: 0.5660, Accuracy: 9878/10000 (98.78%)

New best model saved with accuracy: 98.78%
Current learning rate: 0.000938
Train Epoch: 5 [0/60000 (0%)]	Loss: 0.742958
Train Epoch: 5 [12800/60000 (21%)]	Loss: 0.738377
Train Epoch: 5 [25600/60000 (43%)]	Loss: 0.738249
Train Epoch: 5 [38400/60000 (64%)]	Loss: 0.724232
Train Epoch: 5 [51200/60000 (85%)]	Loss: 0.673112
Train Epoch: 5 - Loss: 0.7522, Accuracy: 90.35%

Test set: Average loss: 0.5610, Accuracy: 9881/10000 (98.81%)

New best model saved with accuracy: 98.81%
Current learning rate: 0.000905
Train Epoch: 6 [0/60000 (0%)]	Loss: 0.832365
Train Epoch: 6 [12800/60000 (21%)]	Loss: 0.742722
Train Epoch: 6 [25600/60000 (43%)]	Loss: 0.719453
Train Epoch: 6 [38400/60000 (64%)]	Loss: 0.690084
Train Epoch: 6 [51200/60000 (85%)]	Loss: 0.731918
Train Epoch: 6 - Loss: 0.7376, Accuracy: 91.00%

Test set: Average loss: 0.5504, Accuracy: 9901/10000 (99.01%)

New best model saved with accuracy: 99.01%
Current learning rate: 0.000864
Train Epoch: 7 [0/60000 (0%)]	Loss: 0.797562
Train Epoch: 7 [12800/60000 (21%)]	Loss: 0.704427
Train Epoch: 7 [25600/60000 (43%)]	Loss: 0.751022
Train Epoch: 7 [38400/60000 (64%)]	Loss: 0.722858
Train Epoch: 7 [51200/60000 (85%)]	Loss: 0.779217
Train Epoch: 7 - Loss: 0.7214, Accuracy: 91.60%

Test set: Average loss: 0.5427, Accuracy: 9917/10000 (99.17%)

New best model saved with accuracy: 99.17%
Current learning rate: 0.000819
Train Epoch: 8 [0/60000 (0%)]	Loss: 0.725445
Train Epoch: 8 [12800/60000 (21%)]	Loss: 0.707736
Train Epoch: 8 [25600/60000 (43%)]	Loss: 0.681101
Train Epoch: 8 [38400/60000 (64%)]	Loss: 0.798273
Train Epoch: 8 [51200/60000 (85%)]	Loss: 0.662572
Train Epoch: 8 - Loss: 0.7115, Accuracy: 92.00%

Test set: Average loss: 0.5440, Accuracy: 9915/10000 (99.15%)

Current learning rate: 0.000768
Train Epoch: 9 [0/60000 (0%)]	Loss: 0.670381
Train Epoch: 9 [12800/60000 (21%)]	Loss: 0.785942
Train Epoch: 9 [25600/60000 (43%)]	Loss: 0.676740
Train Epoch: 9 [38400/60000 (64%)]	Loss: 0.719574
Train Epoch: 9 [51200/60000 (85%)]	Loss: 0.691495
Train Epoch: 9 - Loss: 0.7035, Accuracy: 92.18%

Test set: Average loss: 0.5406, Accuracy: 9922/10000 (99.22%)

New best model saved with accuracy: 99.22%
Current learning rate: 0.000713
Train Epoch: 10 [0/60000 (0%)]	Loss: 0.655587
Train Epoch: 10 [12800/60000 (21%)]	Loss: 0.687294
Train Epoch: 10 [25600/60000 (43%)]	Loss: 0.818706
Train Epoch: 10 [38400/60000 (64%)]	Loss: 0.672551
Train Epoch: 10 [51200/60000 (85%)]	Loss: 0.694415
Train Epoch: 10 - Loss: 0.6996, Accuracy: 92.39%

Test set: Average loss: 0.5394, Accuracy: 9917/10000 (99.17%)

Current learning rate: 0.000655
Train Epoch: 11 [0/60000 (0%)]	Loss: 0.671438
Train Epoch: 11 [12800/60000 (21%)]	Loss: 0.722561
Train Epoch: 11 [25600/60000 (43%)]	Loss: 0.732145
Train Epoch: 11 [38400/60000 (64%)]	Loss: 0.720372
Train Epoch: 11 [51200/60000 (85%)]	Loss: 0.706968
Train Epoch: 11 - Loss: 0.6899, Accuracy: 92.70%

Test set: Average loss: 0.5366, Accuracy: 9927/10000 (99.27%)

New best model saved with accuracy: 99.27%
Current learning rate: 0.000594
Train Epoch: 12 [0/60000 (0%)]	Loss: 0.714624
Train Epoch: 12 [12800/60000 (21%)]	Loss: 0.667079
Train Epoch: 12 [25600/60000 (43%)]	Loss: 0.675105
Train Epoch: 12 [38400/60000 (64%)]	Loss: 0.667722
Train Epoch: 12 [51200/60000 (85%)]	Loss: 0.657083
Train Epoch: 12 - Loss: 0.6872, Accuracy: 92.89%

Test set: Average loss: 0.5338, Accuracy: 9930/10000 (99.30%)

New best model saved with accuracy: 99.30%
Current learning rate: 0.000531
Train Epoch: 13 [0/60000 (0%)]	Loss: 0.624387
Train Epoch: 13 [12800/60000 (21%)]	Loss: 0.607477
Train Epoch: 13 [25600/60000 (43%)]	Loss: 0.732681
Train Epoch: 13 [38400/60000 (64%)]	Loss: 0.707490
Train Epoch: 13 [51200/60000 (85%)]	Loss: 0.672472
Train Epoch: 13 - Loss: 0.6839, Accuracy: 92.83%

Test set: Average loss: 0.5341, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000469
Train Epoch: 14 [0/60000 (0%)]	Loss: 0.701805
Train Epoch: 14 [12800/60000 (21%)]	Loss: 0.645073
Train Epoch: 14 [25600/60000 (43%)]	Loss: 0.664362
Train Epoch: 14 [38400/60000 (64%)]	Loss: 0.706534
Train Epoch: 14 [51200/60000 (85%)]	Loss: 0.672159
Train Epoch: 14 - Loss: 0.6797, Accuracy: 93.07%

Test set: Average loss: 0.5333, Accuracy: 9929/10000 (99.29%)

Current learning rate: 0.000406
Train Epoch: 15 [0/60000 (0%)]	Loss: 0.608367
Train Epoch: 15 [12800/60000 (21%)]	Loss: 0.680665
Train Epoch: 15 [25600/60000 (43%)]	Loss: 0.621114
Train Epoch: 15 [38400/60000 (64%)]	Loss: 0.749492
Train Epoch: 15 [51200/60000 (85%)]	Loss: 0.604757
Train Epoch: 15 - Loss: 0.6757, Accuracy: 93.22%

Test set: Average loss: 0.5325, Accuracy: 9934/10000 (99.34%)

New best model saved with accuracy: 99.34%
Current learning rate: 0.000345
Train Epoch: 16 [0/60000 (0%)]	Loss: 0.639956
Train Epoch: 16 [12800/60000 (21%)]	Loss: 0.613618
Train Epoch: 16 [25600/60000 (43%)]	Loss: 0.701154
Train Epoch: 16 [38400/60000 (64%)]	Loss: 0.687948
Train Epoch: 16 [51200/60000 (85%)]	Loss: 0.677372
Train Epoch: 16 - Loss: 0.6746, Accuracy: 93.24%

Test set: Average loss: 0.5315, Accuracy: 9927/10000 (99.27%)

Current learning rate: 0.000287
Train Epoch: 17 [0/60000 (0%)]	Loss: 0.634485
Train Epoch: 17 [12800/60000 (21%)]	Loss: 0.616510
Train Epoch: 17 [25600/60000 (43%)]	Loss: 0.617930
Train Epoch: 17 [38400/60000 (64%)]	Loss: 0.614425
Train Epoch: 17 [51200/60000 (85%)]	Loss: 0.660623
Train Epoch: 17 - Loss: 0.6749, Accuracy: 93.16%

Test set: Average loss: 0.5306, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000232
Train Epoch: 18 [0/60000 (0%)]	Loss: 0.675041
Train Epoch: 18 [12800/60000 (21%)]	Loss: 0.653175
Train Epoch: 18 [25600/60000 (43%)]	Loss: 0.670096
Train Epoch: 18 [38400/60000 (64%)]	Loss: 0.656966
Train Epoch: 18 [51200/60000 (85%)]	Loss: 0.682921
Train Epoch: 18 - Loss: 0.6702, Accuracy: 93.47%

Test set: Average loss: 0.5307, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000181
Train Epoch: 19 [0/60000 (0%)]	Loss: 0.658921
Train Epoch: 19 [12800/60000 (21%)]	Loss: 0.664013
Train Epoch: 19 [25600/60000 (43%)]	Loss: 0.711457
Train Epoch: 19 [38400/60000 (64%)]	Loss: 0.666313
Train Epoch: 19 [51200/60000 (85%)]	Loss: 0.622270
Train Epoch: 19 - Loss: 0.6690, Accuracy: 93.46%

Test set: Average loss: 0.5294, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000136
Train Epoch: 20 [0/60000 (0%)]	Loss: 0.663213
Train Epoch: 20 [12800/60000 (21%)]	Loss: 0.700877
Train Epoch: 20 [25600/60000 (43%)]	Loss: 0.640258
Train Epoch: 20 [38400/60000 (64%)]	Loss: 0.680927
Train Epoch: 20 [51200/60000 (85%)]	Loss: 0.605670
Train Epoch: 20 - Loss: 0.6678, Accuracy: 93.47%

Test set: Average loss: 0.5285, Accuracy: 9932/10000 (99.32%)

Current learning rate: 0.000095
Train Epoch: 21 [0/60000 (0%)]	Loss: 0.661894
Train Epoch: 21 [12800/60000 (21%)]	Loss: 0.700235
Train Epoch: 21 [25600/60000 (43%)]	Loss: 0.608073
Train Epoch: 21 [38400/60000 (64%)]	Loss: 0.624671
Train Epoch: 21 [51200/60000 (85%)]	Loss: 0.675810
Train Epoch: 21 - Loss: 0.6691, Accuracy: 93.42%

Test set: Average loss: 0.5287, Accuracy: 9934/10000 (99.34%)

Current learning rate: 0.000062
Train Epoch: 22 [0/60000 (0%)]	Loss: 0.639909
Train Epoch: 22 [12800/60000 (21%)]	Loss: 0.692506
Train Epoch: 22 [25600/60000 (43%)]	Loss: 0.641128
Train Epoch: 22 [38400/60000 (64%)]	Loss: 0.686204
Train Epoch: 22 [51200/60000 (85%)]	Loss: 0.613364
Train Epoch: 22 - Loss: 0.6655, Accuracy: 93.62%

Test set: Average loss: 0.5294, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000035
Train Epoch: 23 [0/60000 (0%)]	Loss: 0.628520
Train Epoch: 23 [12800/60000 (21%)]	Loss: 0.751516
Train Epoch: 23 [25600/60000 (43%)]	Loss: 0.648665
Train Epoch: 23 [38400/60000 (64%)]	Loss: 0.636809
Train Epoch: 23 [51200/60000 (85%)]	Loss: 0.596888
Train Epoch: 23 - Loss: 0.6651, Accuracy: 93.61%

Test set: Average loss: 0.5294, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000016
Train Epoch: 24 [0/60000 (0%)]	Loss: 0.653868
Train Epoch: 24 [12800/60000 (21%)]	Loss: 0.650533
Train Epoch: 24 [25600/60000 (43%)]	Loss: 0.614806
Train Epoch: 24 [38400/60000 (64%)]	Loss: 0.690783
Train Epoch: 24 [51200/60000 (85%)]	Loss: 0.646161
Train Epoch: 24 - Loss: 0.6632, Accuracy: 93.66%

Test set: Average loss: 0.5287, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000004
Train Epoch: 25 [0/60000 (0%)]	Loss: 0.646714
Train Epoch: 25 [12800/60000 (21%)]	Loss: 0.720124
Train Epoch: 25 [25600/60000 (43%)]	Loss: 0.648952
Train Epoch: 25 [38400/60000 (64%)]	Loss: 0.667014
Train Epoch: 25 [51200/60000 (85%)]	Loss: 0.669611
Train Epoch: 25 - Loss: 0.6669, Accuracy: 93.50%

Test set: Average loss: 0.5290, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.001000
Train Epoch: 26 [0/60000 (0%)]	Loss: 0.685317
Train Epoch: 26 [12800/60000 (21%)]	Loss: 0.629416
Train Epoch: 26 [25600/60000 (43%)]	Loss: 0.710505
Train Epoch: 26 [38400/60000 (64%)]	Loss: 0.657811
Train Epoch: 26 [51200/60000 (85%)]	Loss: 0.641703
Train Epoch: 26 - Loss: 0.6762, Accuracy: 93.18%

Test set: Average loss: 0.5307, Accuracy: 9925/10000 (99.25%)

Current learning rate: 0.000996
Train Epoch: 27 [0/60000 (0%)]	Loss: 0.675804
Train Epoch: 27 [12800/60000 (21%)]	Loss: 0.685089
Train Epoch: 27 [25600/60000 (43%)]	Loss: 0.641297
Train Epoch: 27 [38400/60000 (64%)]	Loss: 0.636269
Train Epoch: 27 [51200/60000 (85%)]	Loss: 0.752571
Train Epoch: 27 - Loss: 0.6733, Accuracy: 93.25%

Test set: Average loss: 0.5313, Accuracy: 9941/10000 (99.41%)

New best model saved with accuracy: 99.41%
Current learning rate: 0.000984
Train Epoch: 28 [0/60000 (0%)]	Loss: 0.683434
Train Epoch: 28 [12800/60000 (21%)]	Loss: 0.683293
Train Epoch: 28 [25600/60000 (43%)]	Loss: 0.697354
Train Epoch: 28 [38400/60000 (64%)]	Loss: 0.665240
Train Epoch: 28 [51200/60000 (85%)]	Loss: 0.712986
Train Epoch: 28 - Loss: 0.6736, Accuracy: 93.17%

Test set: Average loss: 0.5310, Accuracy: 9931/10000 (99.31%)

Current learning rate: 0.000965
Train Epoch: 29 [0/60000 (0%)]	Loss: 0.646743
Train Epoch: 29 [12800/60000 (21%)]	Loss: 0.671196
Train Epoch: 29 [25600/60000 (43%)]	Loss: 0.690846
Train Epoch: 29 [38400/60000 (64%)]	Loss: 0.663021
Train Epoch: 29 [51200/60000 (85%)]	Loss: 0.669096
Train Epoch: 29 - Loss: 0.6676, Accuracy: 93.44%

Test set: Average loss: 0.5303, Accuracy: 9928/10000 (99.28%)

Current learning rate: 0.000938
Train Epoch: 30 [0/60000 (0%)]	Loss: 0.606382
Train Epoch: 30 [12800/60000 (21%)]	Loss: 0.653498
Train Epoch: 30 [25600/60000 (43%)]	Loss: 0.644413
Train Epoch: 30 [38400/60000 (64%)]	Loss: 0.701360
Train Epoch: 30 [51200/60000 (85%)]	Loss: 0.808356
Train Epoch: 30 - Loss: 0.6666, Accuracy: 93.45%

Test set: Average loss: 0.5283, Accuracy: 9933/10000 (99.33%)

Current learning rate: 0.000905
Train Epoch: 31 [0/60000 (0%)]	Loss: 0.625708
Train Epoch: 31 [12800/60000 (21%)]	Loss: 0.676620
Train Epoch: 31 [25600/60000 (43%)]	Loss: 0.671144
Train Epoch: 31 [38400/60000 (64%)]	Loss: 0.634990
Train Epoch: 31 [51200/60000 (85%)]	Loss: 0.657511
Train Epoch: 31 - Loss: 0.6619, Accuracy: 93.81%

Test set: Average loss: 0.5267, Accuracy: 9944/10000 (99.44%)

New best model saved with accuracy: 99.44%
Current learning rate: 0.000864
Train Epoch: 32 [0/60000 (0%)]	Loss: 0.716368
Train Epoch: 32 [12800/60000 (21%)]	Loss: 0.759162
Train Epoch: 32 [25600/60000 (43%)]	Loss: 0.580527
Train Epoch: 32 [38400/60000 (64%)]	Loss: 0.712267
Train Epoch: 32 [51200/60000 (85%)]	Loss: 0.656820
Train Epoch: 32 - Loss: 0.6619, Accuracy: 93.65%

Test set: Average loss: 0.5259, Accuracy: 9939/10000 (99.39%)

Current learning rate: 0.000819
Train Epoch: 33 [0/60000 (0%)]	Loss: 0.643773
Train Epoch: 33 [12800/60000 (21%)]	Loss: 0.658479
Train Epoch: 33 [25600/60000 (43%)]	Loss: 0.606511
Train Epoch: 33 [38400/60000 (64%)]	Loss: 0.725166
Train Epoch: 33 [51200/60000 (85%)]	Loss: 0.666851
Train Epoch: 33 - Loss: 0.6591, Accuracy: 93.72%

Test set: Average loss: 0.5268, Accuracy: 9940/10000 (99.40%)

Current learning rate: 0.000768
Train Epoch: 34 [0/60000 (0%)]	Loss: 0.705684
Train Epoch: 34 [12800/60000 (21%)]	Loss: 0.635782
Train Epoch: 34 [25600/60000 (43%)]	Loss: 0.617923
Train Epoch: 34 [38400/60000 (64%)]	Loss: 0.672518
Train Epoch: 34 [51200/60000 (85%)]	Loss: 0.642843
Train Epoch: 34 - Loss: 0.6575, Accuracy: 93.74%

Test set: Average loss: 0.5250, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000713
Train Epoch: 35 [0/60000 (0%)]	Loss: 0.616217
Train Epoch: 35 [12800/60000 (21%)]	Loss: 0.618298
Train Epoch: 35 [25600/60000 (43%)]	Loss: 0.672238
Train Epoch: 35 [38400/60000 (64%)]	Loss: 0.715126
Train Epoch: 35 [51200/60000 (85%)]	Loss: 0.649182
Train Epoch: 35 - Loss: 0.6551, Accuracy: 93.92%

Test set: Average loss: 0.5241, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000655
Train Epoch: 36 [0/60000 (0%)]	Loss: 0.662871
Train Epoch: 36 [12800/60000 (21%)]	Loss: 0.684718
Train Epoch: 36 [25600/60000 (43%)]	Loss: 0.579904
Train Epoch: 36 [38400/60000 (64%)]	Loss: 0.643077
Train Epoch: 36 [51200/60000 (85%)]	Loss: 0.615307
Train Epoch: 36 - Loss: 0.6540, Accuracy: 93.95%

Test set: Average loss: 0.5230, Accuracy: 9946/10000 (99.46%)

New best model saved with accuracy: 99.46%
Current learning rate: 0.000594
Train Epoch: 37 [0/60000 (0%)]	Loss: 0.654092
Train Epoch: 37 [12800/60000 (21%)]	Loss: 0.653188
Train Epoch: 37 [25600/60000 (43%)]	Loss: 0.643334
Train Epoch: 37 [38400/60000 (64%)]	Loss: 0.699037
Train Epoch: 37 [51200/60000 (85%)]	Loss: 0.626194
Train Epoch: 37 - Loss: 0.6534, Accuracy: 93.97%

Test set: Average loss: 0.5229, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000531
Train Epoch: 38 [0/60000 (0%)]	Loss: 0.597758
Train Epoch: 38 [12800/60000 (21%)]	Loss: 0.672243
Train Epoch: 38 [25600/60000 (43%)]	Loss: 0.638330
Train Epoch: 38 [38400/60000 (64%)]	Loss: 0.625927
Train Epoch: 38 [51200/60000 (85%)]	Loss: 0.626993
Train Epoch: 38 - Loss: 0.6490, Accuracy: 93.99%

Test set: Average loss: 0.5240, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000469
Train Epoch: 39 [0/60000 (0%)]	Loss: 0.645614
Train Epoch: 39 [12800/60000 (21%)]	Loss: 0.726393
Train Epoch: 39 [25600/60000 (43%)]	Loss: 0.655338
Train Epoch: 39 [38400/60000 (64%)]	Loss: 0.626931
Train Epoch: 39 [51200/60000 (85%)]	Loss: 0.672785
Train Epoch: 39 - Loss: 0.6478, Accuracy: 94.15%

Test set: Average loss: 0.5222, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000406
Train Epoch: 40 [0/60000 (0%)]	Loss: 0.688774
Train Epoch: 40 [12800/60000 (21%)]	Loss: 0.613507
Train Epoch: 40 [25600/60000 (43%)]	Loss: 0.598103
Train Epoch: 40 [38400/60000 (64%)]	Loss: 0.662048
Train Epoch: 40 [51200/60000 (85%)]	Loss: 0.701018
Train Epoch: 40 - Loss: 0.6459, Accuracy: 94.23%

Test set: Average loss: 0.5222, Accuracy: 9947/10000 (99.47%)

New best model saved with accuracy: 99.47%
Current learning rate: 0.000345
Train Epoch: 41 [0/60000 (0%)]	Loss: 0.666669
Train Epoch: 41 [12800/60000 (21%)]	Loss: 0.654106
Train Epoch: 41 [25600/60000 (43%)]	Loss: 0.608200
Train Epoch: 41 [38400/60000 (64%)]	Loss: 0.616964
Train Epoch: 41 [51200/60000 (85%)]	Loss: 0.654077
Train Epoch: 41 - Loss: 0.6479, Accuracy: 94.16%

Test set: Average loss: 0.5215, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000287
Train Epoch: 42 [0/60000 (0%)]	Loss: 0.631104
Train Epoch: 42 [12800/60000 (21%)]	Loss: 0.661115
Train Epoch: 42 [25600/60000 (43%)]	Loss: 0.638069
Train Epoch: 42 [38400/60000 (64%)]	Loss: 0.618220
Train Epoch: 42 [51200/60000 (85%)]	Loss: 0.608791
Train Epoch: 42 - Loss: 0.6458, Accuracy: 94.15%

Test set: Average loss: 0.5218, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000232
Train Epoch: 43 [0/60000 (0%)]	Loss: 0.648119
Train Epoch: 43 [12800/60000 (21%)]	Loss: 0.720860
Train Epoch: 43 [25600/60000 (43%)]	Loss: 0.613626
Train Epoch: 43 [38400/60000 (64%)]	Loss: 0.638774
Train Epoch: 43 [51200/60000 (85%)]	Loss: 0.564779
Train Epoch: 43 - Loss: 0.6412, Accuracy: 94.38%

Test set: Average loss: 0.5211, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000181
Train Epoch: 44 [0/60000 (0%)]	Loss: 0.657401
Train Epoch: 44 [12800/60000 (21%)]	Loss: 0.678700
Train Epoch: 44 [25600/60000 (43%)]	Loss: 0.636953
Train Epoch: 44 [38400/60000 (64%)]	Loss: 0.597605
Train Epoch: 44 [51200/60000 (85%)]	Loss: 0.732778
Train Epoch: 44 - Loss: 0.6421, Accuracy: 94.36%

Test set: Average loss: 0.5211, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000136
Train Epoch: 45 [0/60000 (0%)]	Loss: 0.657398
Train Epoch: 45 [12800/60000 (21%)]	Loss: 0.601095
Train Epoch: 45 [25600/60000 (43%)]	Loss: 0.631149
Train Epoch: 45 [38400/60000 (64%)]	Loss: 0.622557
Train Epoch: 45 [51200/60000 (85%)]	Loss: 0.685448
Train Epoch: 45 - Loss: 0.6426, Accuracy: 94.37%

Test set: Average loss: 0.5208, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000095
Train Epoch: 46 [0/60000 (0%)]	Loss: 0.668088
Train Epoch: 46 [12800/60000 (21%)]	Loss: 0.601210
Train Epoch: 46 [25600/60000 (43%)]	Loss: 0.627794
Train Epoch: 46 [38400/60000 (64%)]	Loss: 0.633564
Train Epoch: 46 [51200/60000 (85%)]	Loss: 0.667385
Train Epoch: 46 - Loss: 0.6405, Accuracy: 94.50%

Test set: Average loss: 0.5213, Accuracy: 9942/10000 (99.42%)

Current learning rate: 0.000062
Train Epoch: 47 [0/60000 (0%)]	Loss: 0.656510
Train Epoch: 47 [12800/60000 (21%)]	Loss: 0.615136
Train Epoch: 47 [25600/60000 (43%)]	Loss: 0.709537
Train Epoch: 47 [38400/60000 (64%)]	Loss: 0.605468
Train Epoch: 47 [51200/60000 (85%)]	Loss: 0.647831
Train Epoch: 47 - Loss: 0.6410, Accuracy: 94.33%

Test set: Average loss: 0.5207, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000035
Train Epoch: 48 [0/60000 (0%)]	Loss: 0.629943
Train Epoch: 48 [12800/60000 (21%)]	Loss: 0.624894
Train Epoch: 48 [25600/60000 (43%)]	Loss: 0.672895
Train Epoch: 48 [38400/60000 (64%)]	Loss: 0.672936
Train Epoch: 48 [51200/60000 (85%)]	Loss: 0.631515
Train Epoch: 48 - Loss: 0.6401, Accuracy: 94.43%

Test set: Average loss: 0.5208, Accuracy: 9948/10000 (99.48%)

New best model saved with accuracy: 99.48%
Current learning rate: 0.000016
Train Epoch: 49 [0/60000 (0%)]	Loss: 0.621754
Train Epoch: 49 [12800/60000 (21%)]	Loss: 0.633120
Train Epoch: 49 [25600/60000 (43%)]	Loss: 0.615712
Train Epoch: 49 [38400/60000 (64%)]	Loss: 0.608525
Train Epoch: 49 [51200/60000 (85%)]	Loss: 0.644695
Train Epoch: 49 - Loss: 0.6417, Accuracy: 94.30%

Test set: Average loss: 0.5212, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000004
Train Epoch: 50 [0/60000 (0%)]	Loss: 0.636855
Train Epoch: 50 [12800/60000 (21%)]	Loss: 0.615966
Train Epoch: 50 [25600/60000 (43%)]	Loss: 0.659266
Train Epoch: 50 [38400/60000 (64%)]	Loss: 0.586317
Train Epoch: 50 [51200/60000 (85%)]	Loss: 0.598768
Train Epoch: 50 - Loss: 0.6380, Accuracy: 94.55%

Test set: Average loss: 0.5207, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.001000
Train Epoch: 51 [0/60000 (0%)]	Loss: 0.584786
Train Epoch: 51 [12800/60000 (21%)]	Loss: 0.712716
Train Epoch: 51 [25600/60000 (43%)]	Loss: 0.671888
Train Epoch: 51 [38400/60000 (64%)]	Loss: 0.630735
Train Epoch: 51 [51200/60000 (85%)]	Loss: 0.620146
Train Epoch: 51 - Loss: 0.6513, Accuracy: 93.93%

Test set: Average loss: 0.5228, Accuracy: 9938/10000 (99.38%)

Current learning rate: 0.000996
Train Epoch: 52 [0/60000 (0%)]	Loss: 0.606371
Train Epoch: 52 [12800/60000 (21%)]	Loss: 0.679178
Train Epoch: 52 [25600/60000 (43%)]	Loss: 0.687689
Train Epoch: 52 [38400/60000 (64%)]	Loss: 0.635707
Train Epoch: 52 [51200/60000 (85%)]	Loss: 0.634671
Train Epoch: 52 - Loss: 0.6497, Accuracy: 94.08%

Test set: Average loss: 0.5239, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000984
Train Epoch: 53 [0/60000 (0%)]	Loss: 0.636000
Train Epoch: 53 [12800/60000 (21%)]	Loss: 0.635457
Train Epoch: 53 [25600/60000 (43%)]	Loss: 0.619737
Train Epoch: 53 [38400/60000 (64%)]	Loss: 0.669431
Train Epoch: 53 [51200/60000 (85%)]	Loss: 0.643303
Train Epoch: 53 - Loss: 0.6497, Accuracy: 94.08%

Test set: Average loss: 0.5224, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000965
Train Epoch: 54 [0/60000 (0%)]	Loss: 0.635743
Train Epoch: 54 [12800/60000 (21%)]	Loss: 0.705019
Train Epoch: 54 [25600/60000 (43%)]	Loss: 0.649758
Train Epoch: 54 [38400/60000 (64%)]	Loss: 0.683162
Train Epoch: 54 [51200/60000 (85%)]	Loss: 0.662029
Train Epoch: 54 - Loss: 0.6461, Accuracy: 94.15%

Test set: Average loss: 0.5216, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000938
Train Epoch: 55 [0/60000 (0%)]	Loss: 0.686214
Train Epoch: 55 [12800/60000 (21%)]	Loss: 0.646404
Train Epoch: 55 [25600/60000 (43%)]	Loss: 0.648629
Train Epoch: 55 [38400/60000 (64%)]	Loss: 0.704396
Train Epoch: 55 [51200/60000 (85%)]	Loss: 0.672617
Train Epoch: 55 - Loss: 0.6480, Accuracy: 94.08%

Test set: Average loss: 0.5229, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000905
Train Epoch: 56 [0/60000 (0%)]	Loss: 0.613021
Train Epoch: 56 [12800/60000 (21%)]	Loss: 0.598679
Train Epoch: 56 [25600/60000 (43%)]	Loss: 0.683005
Train Epoch: 56 [38400/60000 (64%)]	Loss: 0.641829
Train Epoch: 56 [51200/60000 (85%)]	Loss: 0.716872
Train Epoch: 56 - Loss: 0.6467, Accuracy: 94.09%

Test set: Average loss: 0.5219, Accuracy: 9943/10000 (99.43%)

Current learning rate: 0.000864
Train Epoch: 57 [0/60000 (0%)]	Loss: 0.607715
Train Epoch: 57 [12800/60000 (21%)]	Loss: 0.629066
Train Epoch: 57 [25600/60000 (43%)]	Loss: 0.692131
Train Epoch: 57 [38400/60000 (64%)]	Loss: 0.609324
Train Epoch: 57 [51200/60000 (85%)]	Loss: 0.666009
Train Epoch: 57 - Loss: 0.6435, Accuracy: 94.27%

Test set: Average loss: 0.5212, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000819
Train Epoch: 58 [0/60000 (0%)]	Loss: 0.596842
Train Epoch: 58 [12800/60000 (21%)]	Loss: 0.693435
Train Epoch: 58 [25600/60000 (43%)]	Loss: 0.604813
Train Epoch: 58 [38400/60000 (64%)]	Loss: 0.614624
Train Epoch: 58 [51200/60000 (85%)]	Loss: 0.677751
Train Epoch: 58 - Loss: 0.6417, Accuracy: 94.45%

Test set: Average loss: 0.5208, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000768
Train Epoch: 59 [0/60000 (0%)]	Loss: 0.645738
Train Epoch: 59 [12800/60000 (21%)]	Loss: 0.617657
Train Epoch: 59 [25600/60000 (43%)]	Loss: 0.612305
Train Epoch: 59 [38400/60000 (64%)]	Loss: 0.655953
Train Epoch: 59 [51200/60000 (85%)]	Loss: 0.640771
Train Epoch: 59 - Loss: 0.6406, Accuracy: 94.43%

Test set: Average loss: 0.5216, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000713
Train Epoch: 60 [0/60000 (0%)]	Loss: 0.644923
Train Epoch: 60 [12800/60000 (21%)]	Loss: 0.653779
Train Epoch: 60 [25600/60000 (43%)]	Loss: 0.667364
Train Epoch: 60 [38400/60000 (64%)]	Loss: 0.672500
Train Epoch: 60 [51200/60000 (85%)]	Loss: 0.623697
Train Epoch: 60 - Loss: 0.6406, Accuracy: 94.37%

Test set: Average loss: 0.5201, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000655
Train Epoch: 61 [0/60000 (0%)]	Loss: 0.718952
Train Epoch: 61 [12800/60000 (21%)]	Loss: 0.635305
Train Epoch: 61 [25600/60000 (43%)]	Loss: 0.564790
Train Epoch: 61 [38400/60000 (64%)]	Loss: 0.652399
Train Epoch: 61 [51200/60000 (85%)]	Loss: 0.619169
Train Epoch: 61 - Loss: 0.6402, Accuracy: 94.46%

Test set: Average loss: 0.5193, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000594
Train Epoch: 62 [0/60000 (0%)]	Loss: 0.620800
Train Epoch: 62 [12800/60000 (21%)]	Loss: 0.588837
Train Epoch: 62 [25600/60000 (43%)]	Loss: 0.675436
Train Epoch: 62 [38400/60000 (64%)]	Loss: 0.622743
Train Epoch: 62 [51200/60000 (85%)]	Loss: 0.638177
Train Epoch: 62 - Loss: 0.6407, Accuracy: 94.36%

Test set: Average loss: 0.5203, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000531
Train Epoch: 63 [0/60000 (0%)]	Loss: 0.638391
Train Epoch: 63 [12800/60000 (21%)]	Loss: 0.616702
Train Epoch: 63 [25600/60000 (43%)]	Loss: 0.639683
Train Epoch: 63 [38400/60000 (64%)]	Loss: 0.588938
Train Epoch: 63 [51200/60000 (85%)]	Loss: 0.645889
Train Epoch: 63 - Loss: 0.6360, Accuracy: 94.48%

Test set: Average loss: 0.5183, Accuracy: 9952/10000 (99.52%)

New best model saved with accuracy: 99.52%
Current learning rate: 0.000469
Train Epoch: 64 [0/60000 (0%)]	Loss: 0.605741
Train Epoch: 64 [12800/60000 (21%)]	Loss: 0.659463
Train Epoch: 64 [25600/60000 (43%)]	Loss: 0.650497
Train Epoch: 64 [38400/60000 (64%)]	Loss: 0.606963
Train Epoch: 64 [51200/60000 (85%)]	Loss: 0.715090
Train Epoch: 64 - Loss: 0.6367, Accuracy: 94.42%

Test set: Average loss: 0.5197, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000406
Train Epoch: 65 [0/60000 (0%)]	Loss: 0.649361
Train Epoch: 65 [12800/60000 (21%)]	Loss: 0.655225
Train Epoch: 65 [25600/60000 (43%)]	Loss: 0.638016
Train Epoch: 65 [38400/60000 (64%)]	Loss: 0.610238
Train Epoch: 65 [51200/60000 (85%)]	Loss: 0.605905
Train Epoch: 65 - Loss: 0.6324, Accuracy: 94.68%

Test set: Average loss: 0.5188, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000345
Train Epoch: 66 [0/60000 (0%)]	Loss: 0.583530
Train Epoch: 66 [12800/60000 (21%)]	Loss: 0.629946
Train Epoch: 66 [25600/60000 (43%)]	Loss: 0.661109
Train Epoch: 66 [38400/60000 (64%)]	Loss: 0.700318
Train Epoch: 66 [51200/60000 (85%)]	Loss: 0.600646
Train Epoch: 66 - Loss: 0.6309, Accuracy: 94.74%

Test set: Average loss: 0.5192, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000287
Train Epoch: 67 [0/60000 (0%)]	Loss: 0.618267
Train Epoch: 67 [12800/60000 (21%)]	Loss: 0.614484
Train Epoch: 67 [25600/60000 (43%)]	Loss: 0.627492
Train Epoch: 67 [38400/60000 (64%)]	Loss: 0.663549
Train Epoch: 67 [51200/60000 (85%)]	Loss: 0.642892
Train Epoch: 67 - Loss: 0.6323, Accuracy: 94.73%

Test set: Average loss: 0.5188, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000232
Train Epoch: 68 [0/60000 (0%)]	Loss: 0.690043
Train Epoch: 68 [12800/60000 (21%)]	Loss: 0.586348
Train Epoch: 68 [25600/60000 (43%)]	Loss: 0.584819
Train Epoch: 68 [38400/60000 (64%)]	Loss: 0.626798
Train Epoch: 68 [51200/60000 (85%)]	Loss: 0.569500
Train Epoch: 68 - Loss: 0.6324, Accuracy: 94.67%

Test set: Average loss: 0.5191, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000181
Train Epoch: 69 [0/60000 (0%)]	Loss: 0.651600
Train Epoch: 69 [12800/60000 (21%)]	Loss: 0.638319
Train Epoch: 69 [25600/60000 (43%)]	Loss: 0.621901
Train Epoch: 69 [38400/60000 (64%)]	Loss: 0.625644
Train Epoch: 69 [51200/60000 (85%)]	Loss: 0.597863
Train Epoch: 69 - Loss: 0.6328, Accuracy: 94.64%

Test set: Average loss: 0.5179, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000136
Train Epoch: 70 [0/60000 (0%)]	Loss: 0.671612
Train Epoch: 70 [12800/60000 (21%)]	Loss: 0.602840
Train Epoch: 70 [25600/60000 (43%)]	Loss: 0.599286
Train Epoch: 70 [38400/60000 (64%)]	Loss: 0.612910
Train Epoch: 70 [51200/60000 (85%)]	Loss: 0.592130
Train Epoch: 70 - Loss: 0.6300, Accuracy: 94.72%

Test set: Average loss: 0.5179, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000095
Train Epoch: 71 [0/60000 (0%)]	Loss: 0.577086
Train Epoch: 71 [12800/60000 (21%)]	Loss: 0.619833
Train Epoch: 71 [25600/60000 (43%)]	Loss: 0.704225
Train Epoch: 71 [38400/60000 (64%)]	Loss: 0.627392
Train Epoch: 71 [51200/60000 (85%)]	Loss: 0.608455
Train Epoch: 71 - Loss: 0.6288, Accuracy: 94.83%

Test set: Average loss: 0.5178, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000062
Train Epoch: 72 [0/60000 (0%)]	Loss: 0.623280
Train Epoch: 72 [12800/60000 (21%)]	Loss: 0.678245
Train Epoch: 72 [25600/60000 (43%)]	Loss: 0.672756
Train Epoch: 72 [38400/60000 (64%)]	Loss: 0.624477
Train Epoch: 72 [51200/60000 (85%)]	Loss: 0.638666
Train Epoch: 72 - Loss: 0.6301, Accuracy: 94.73%

Test set: Average loss: 0.5179, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000035
Train Epoch: 73 [0/60000 (0%)]	Loss: 0.581477
Train Epoch: 73 [12800/60000 (21%)]	Loss: 0.653396
Train Epoch: 73 [25600/60000 (43%)]	Loss: 0.633102
Train Epoch: 73 [38400/60000 (64%)]	Loss: 0.604334
Train Epoch: 73 [51200/60000 (85%)]	Loss: 0.601428
Train Epoch: 73 - Loss: 0.6299, Accuracy: 94.80%

Test set: Average loss: 0.5179, Accuracy: 9948/10000 (99.48%)

Current learning rate: 0.000016
Train Epoch: 74 [0/60000 (0%)]	Loss: 0.713659
Train Epoch: 74 [12800/60000 (21%)]	Loss: 0.558729
Train Epoch: 74 [25600/60000 (43%)]	Loss: 0.591906
Train Epoch: 74 [38400/60000 (64%)]	Loss: 0.650124
Train Epoch: 74 [51200/60000 (85%)]	Loss: 0.694950
Train Epoch: 74 - Loss: 0.6293, Accuracy: 94.83%

Test set: Average loss: 0.5179, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000004
Train Epoch: 75 [0/60000 (0%)]	Loss: 0.587240
Train Epoch: 75 [12800/60000 (21%)]	Loss: 0.653732
Train Epoch: 75 [25600/60000 (43%)]	Loss: 0.614708
Train Epoch: 75 [38400/60000 (64%)]	Loss: 0.676877
Train Epoch: 75 [51200/60000 (85%)]	Loss: 0.679407
Train Epoch: 75 - Loss: 0.6280, Accuracy: 94.89%

Test set: Average loss: 0.5183, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.001000
Train Epoch: 76 [0/60000 (0%)]	Loss: 0.629646
Train Epoch: 76 [12800/60000 (21%)]	Loss: 0.696973
Train Epoch: 76 [25600/60000 (43%)]	Loss: 0.662647
Train Epoch: 76 [38400/60000 (64%)]	Loss: 0.632592
Train Epoch: 76 [51200/60000 (85%)]	Loss: 0.629767
Train Epoch: 76 - Loss: 0.6408, Accuracy: 94.25%

Test set: Average loss: 0.5194, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000996
Train Epoch: 77 [0/60000 (0%)]	Loss: 0.606040
Train Epoch: 77 [12800/60000 (21%)]	Loss: 0.664737
Train Epoch: 77 [25600/60000 (43%)]	Loss: 0.655643
Train Epoch: 77 [38400/60000 (64%)]	Loss: 0.729501
Train Epoch: 77 [51200/60000 (85%)]	Loss: 0.624558
Train Epoch: 77 - Loss: 0.6372, Accuracy: 94.49%

Test set: Average loss: 0.5201, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000984
Train Epoch: 78 [0/60000 (0%)]	Loss: 0.652165
Train Epoch: 78 [12800/60000 (21%)]	Loss: 0.619787
Train Epoch: 78 [25600/60000 (43%)]	Loss: 0.668441
Train Epoch: 78 [38400/60000 (64%)]	Loss: 0.595027
Train Epoch: 78 [51200/60000 (85%)]	Loss: 0.644233
Train Epoch: 78 - Loss: 0.6388, Accuracy: 94.36%

Test set: Average loss: 0.5195, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000965
Train Epoch: 79 [0/60000 (0%)]	Loss: 0.652776
Train Epoch: 79 [12800/60000 (21%)]	Loss: 0.612892
Train Epoch: 79 [25600/60000 (43%)]	Loss: 0.590703
Train Epoch: 79 [38400/60000 (64%)]	Loss: 0.607548
Train Epoch: 79 [51200/60000 (85%)]	Loss: 0.601520
Train Epoch: 79 - Loss: 0.6377, Accuracy: 94.60%

Test set: Average loss: 0.5197, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000938
Train Epoch: 80 [0/60000 (0%)]	Loss: 0.699372
Train Epoch: 80 [12800/60000 (21%)]	Loss: 0.665971
Train Epoch: 80 [25600/60000 (43%)]	Loss: 0.573679
Train Epoch: 80 [38400/60000 (64%)]	Loss: 0.627094
Train Epoch: 80 [51200/60000 (85%)]	Loss: 0.641501
Train Epoch: 80 - Loss: 0.6364, Accuracy: 94.54%

Test set: Average loss: 0.5192, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000905
Train Epoch: 81 [0/60000 (0%)]	Loss: 0.660097
Train Epoch: 81 [12800/60000 (21%)]	Loss: 0.590262
Train Epoch: 81 [25600/60000 (43%)]	Loss: 0.642461
Train Epoch: 81 [38400/60000 (64%)]	Loss: 0.638934
Train Epoch: 81 [51200/60000 (85%)]	Loss: 0.621771
Train Epoch: 81 - Loss: 0.6376, Accuracy: 94.44%

Test set: Average loss: 0.5191, Accuracy: 9944/10000 (99.44%)

Current learning rate: 0.000864
Train Epoch: 82 [0/60000 (0%)]	Loss: 0.704084
Train Epoch: 82 [12800/60000 (21%)]	Loss: 0.619551
Train Epoch: 82 [25600/60000 (43%)]	Loss: 0.614597
Train Epoch: 82 [38400/60000 (64%)]	Loss: 0.613136
Train Epoch: 82 [51200/60000 (85%)]	Loss: 0.626991
Train Epoch: 82 - Loss: 0.6347, Accuracy: 94.64%

Test set: Average loss: 0.5179, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000819
Train Epoch: 83 [0/60000 (0%)]	Loss: 0.649285
Train Epoch: 83 [12800/60000 (21%)]	Loss: 0.650609
Train Epoch: 83 [25600/60000 (43%)]	Loss: 0.569211
Train Epoch: 83 [38400/60000 (64%)]	Loss: 0.647522
Train Epoch: 83 [51200/60000 (85%)]	Loss: 0.709803
Train Epoch: 83 - Loss: 0.6321, Accuracy: 94.70%

Test set: Average loss: 0.5174, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000768
Train Epoch: 84 [0/60000 (0%)]	Loss: 0.612526
Train Epoch: 84 [12800/60000 (21%)]	Loss: 0.624618
Train Epoch: 84 [25600/60000 (43%)]	Loss: 0.611921
Train Epoch: 84 [38400/60000 (64%)]	Loss: 0.612853
Train Epoch: 84 [51200/60000 (85%)]	Loss: 0.602427
Train Epoch: 84 - Loss: 0.6289, Accuracy: 94.85%

Test set: Average loss: 0.5174, Accuracy: 9949/10000 (99.49%)

Current learning rate: 0.000713
Train Epoch: 85 [0/60000 (0%)]	Loss: 0.651365
Train Epoch: 85 [12800/60000 (21%)]	Loss: 0.660742
Train Epoch: 85 [25600/60000 (43%)]	Loss: 0.619449
Train Epoch: 85 [38400/60000 (64%)]	Loss: 0.612934
Train Epoch: 85 [51200/60000 (85%)]	Loss: 0.609342
Train Epoch: 85 - Loss: 0.6295, Accuracy: 94.85%

Test set: Average loss: 0.5178, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000655
Train Epoch: 86 [0/60000 (0%)]	Loss: 0.630667
Train Epoch: 86 [12800/60000 (21%)]	Loss: 0.598435
Train Epoch: 86 [25600/60000 (43%)]	Loss: 0.683795
Train Epoch: 86 [38400/60000 (64%)]	Loss: 0.643064
Train Epoch: 86 [51200/60000 (85%)]	Loss: 0.648015
Train Epoch: 86 - Loss: 0.6314, Accuracy: 94.73%

Test set: Average loss: 0.5179, Accuracy: 9945/10000 (99.45%)

Current learning rate: 0.000594
Train Epoch: 87 [0/60000 (0%)]	Loss: 0.616173
Train Epoch: 87 [12800/60000 (21%)]	Loss: 0.616695
Train Epoch: 87 [25600/60000 (43%)]	Loss: 0.621909
Train Epoch: 87 [38400/60000 (64%)]	Loss: 0.605716
Train Epoch: 87 [51200/60000 (85%)]	Loss: 0.593415
Train Epoch: 87 - Loss: 0.6280, Accuracy: 94.80%

Test set: Average loss: 0.5179, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000531
Train Epoch: 88 [0/60000 (0%)]	Loss: 0.649394
Train Epoch: 88 [12800/60000 (21%)]	Loss: 0.681643
Train Epoch: 88 [25600/60000 (43%)]	Loss: 0.684279
Train Epoch: 88 [38400/60000 (64%)]	Loss: 0.623725
Train Epoch: 88 [51200/60000 (85%)]	Loss: 0.645037
Train Epoch: 88 - Loss: 0.6285, Accuracy: 94.86%

Test set: Average loss: 0.5174, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000469
Train Epoch: 89 [0/60000 (0%)]	Loss: 0.610777
Train Epoch: 89 [12800/60000 (21%)]	Loss: 0.631390
Train Epoch: 89 [25600/60000 (43%)]	Loss: 0.593123
Train Epoch: 89 [38400/60000 (64%)]	Loss: 0.626611
Train Epoch: 89 [51200/60000 (85%)]	Loss: 0.676246
Train Epoch: 89 - Loss: 0.6269, Accuracy: 94.87%

Test set: Average loss: 0.5176, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000406
Train Epoch: 90 [0/60000 (0%)]	Loss: 0.666476
Train Epoch: 90 [12800/60000 (21%)]	Loss: 0.580270
Train Epoch: 90 [25600/60000 (43%)]	Loss: 0.606273
Train Epoch: 90 [38400/60000 (64%)]	Loss: 0.599162
Train Epoch: 90 [51200/60000 (85%)]	Loss: 0.676440
Train Epoch: 90 - Loss: 0.6253, Accuracy: 94.92%

Test set: Average loss: 0.5170, Accuracy: 9947/10000 (99.47%)

Current learning rate: 0.000345
Train Epoch: 91 [0/60000 (0%)]	Loss: 0.647591
Train Epoch: 91 [12800/60000 (21%)]	Loss: 0.611575
Train Epoch: 91 [25600/60000 (43%)]	Loss: 0.657527
Train Epoch: 91 [38400/60000 (64%)]	Loss: 0.616605
Train Epoch: 91 [51200/60000 (85%)]	Loss: 0.633536
Train Epoch: 91 - Loss: 0.6279, Accuracy: 94.73%

Test set: Average loss: 0.5169, Accuracy: 9946/10000 (99.46%)

Current learning rate: 0.000287
Train Epoch: 92 [0/60000 (0%)]	Loss: 0.593544
Train Epoch: 92 [12800/60000 (21%)]	Loss: 0.706095
Train Epoch: 92 [25600/60000 (43%)]	Loss: 0.609925
Train Epoch: 92 [38400/60000 (64%)]	Loss: 0.627452
Train Epoch: 92 [51200/60000 (85%)]	Loss: 0.596359
Train Epoch: 92 - Loss: 0.6260, Accuracy: 94.88%

Test set: Average loss: 0.5164, Accuracy: 9952/10000 (99.52%)

Current learning rate: 0.000232
Train Epoch: 93 [0/60000 (0%)]	Loss: 0.634332
Train Epoch: 93 [12800/60000 (21%)]	Loss: 0.581178
Train Epoch: 93 [25600/60000 (43%)]	Loss: 0.638605
Train Epoch: 93 [38400/60000 (64%)]	Loss: 0.613831
Train Epoch: 93 [51200/60000 (85%)]	Loss: 0.696855
Train Epoch: 93 - Loss: 0.6252, Accuracy: 94.92%

Test set: Average loss: 0.5165, Accuracy: 9955/10000 (99.55%)

New best model saved with accuracy: 99.55%
Current learning rate: 0.000181
Train Epoch: 94 [0/60000 (0%)]	Loss: 0.628760
Train Epoch: 94 [12800/60000 (21%)]	Loss: 0.582616
Train Epoch: 94 [25600/60000 (43%)]	Loss: 0.717436
Train Epoch: 94 [38400/60000 (64%)]	Loss: 0.614247
Train Epoch: 94 [51200/60000 (85%)]	Loss: 0.642582
Train Epoch: 94 - Loss: 0.6225, Accuracy: 95.03%

Test set: Average loss: 0.5164, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000136
Train Epoch: 95 [0/60000 (0%)]	Loss: 0.619305
Train Epoch: 95 [12800/60000 (21%)]	Loss: 0.617434
Train Epoch: 95 [25600/60000 (43%)]	Loss: 0.617812
Train Epoch: 95 [38400/60000 (64%)]	Loss: 0.659345
Train Epoch: 95 [51200/60000 (85%)]	Loss: 0.663465
Train Epoch: 95 - Loss: 0.6242, Accuracy: 94.98%

Test set: Average loss: 0.5160, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000095
Train Epoch: 96 [0/60000 (0%)]	Loss: 0.635256
Train Epoch: 96 [12800/60000 (21%)]	Loss: 0.665291
Train Epoch: 96 [25600/60000 (43%)]	Loss: 0.633754
Train Epoch: 96 [38400/60000 (64%)]	Loss: 0.606600
Train Epoch: 96 [51200/60000 (85%)]	Loss: 0.591260
Train Epoch: 96 - Loss: 0.6208, Accuracy: 95.06%

Test set: Average loss: 0.5163, Accuracy: 9950/10000 (99.50%)

Current learning rate: 0.000062
Train Epoch: 97 [0/60000 (0%)]	Loss: 0.588637
Train Epoch: 97 [12800/60000 (21%)]	Loss: 0.605078
Train Epoch: 97 [25600/60000 (43%)]	Loss: 0.600905
Train Epoch: 97 [38400/60000 (64%)]	Loss: 0.664460
Train Epoch: 97 [51200/60000 (85%)]	Loss: 0.604596
Train Epoch: 97 - Loss: 0.6209, Accuracy: 95.10%

Test set: Average loss: 0.5161, Accuracy: 9953/10000 (99.53%)

Current learning rate: 0.000035
Train Epoch: 98 [0/60000 (0%)]	Loss: 0.617820
Train Epoch: 98 [12800/60000 (21%)]	Loss: 0.592504
Train Epoch: 98 [25600/60000 (43%)]	Loss: 0.596416
Train Epoch: 98 [38400/60000 (64%)]	Loss: 0.608816
Train Epoch: 98 [51200/60000 (85%)]	Loss: 0.619622
Train Epoch: 98 - Loss: 0.6220, Accuracy: 95.08%

Test set: Average loss: 0.5161, Accuracy: 9954/10000 (99.54%)

Current learning rate: 0.000016
Train Epoch: 99 [0/60000 (0%)]	Loss: 0.651790
Train Epoch: 99 [12800/60000 (21%)]	Loss: 0.594015
Train Epoch: 99 [25600/60000 (43%)]	Loss: 0.590940
Train Epoch: 99 [38400/60000 (64%)]	Loss: 0.680022
Train Epoch: 99 [51200/60000 (85%)]	Loss: 0.552993
Train Epoch: 99 - Loss: 0.6210, Accuracy: 95.10%

Test set: Average loss: 0.5159, Accuracy: 9951/10000 (99.51%)

Current learning rate: 0.000004
Train Epoch: 100 [0/60000 (0%)]	Loss: 0.628466
Train Epoch: 100 [12800/60000 (21%)]	Loss: 0.625341
Train Epoch: 100 [25600/60000 (43%)]	Loss: 0.624041
Train Epoch: 100 [38400/60000 (64%)]	Loss: 0.611850
Train Epoch: 100 [51200/60000 (85%)]	Loss: 0.606790
Train Epoch: 100 - Loss: 0.6171, Accuracy: 95.20%

Test set: Average loss: 0.5163, Accuracy: 9954/10000 (99.54%)


Training completed. Best Accuracy: 99.55%
