Entraînement de modèle de segmentation
Objectif
On essaie de créer un modèle de segmentation plus performant de lectau1_8 pour les pages de répertoires dont les lignes sont plus serrées - on veut en finir avec les soupes de nouilles !
Données
- les données sont issues de plusieurs fichiers
- création d'un nouveau document (400) sur eScriptorium - contenant au total 267 images.
via eScriptorium
- problème lors de l'import des images (pb rapporté ici : scripta/escriptorium#314 (closed))
sur serveur avec ketos
- on contourne le problème d'import, puisque les fichiers sont complets et on passe en CLI sur le serveur
"from scratch"
ketos segtrain -o seg2_m1 -f page -t allxmls.xmllist --device cuda:1 --augment --threads 64
...
Creating model [1,1200,0,3 Cr7,7,64,2,2 Gn32 Cr3,3,128,2,2 Gn32 Cr3,3,128 Gn32 Cr3,3,256 Gn32 Cr3,3,256 Gn32 Lbx32 Lby32 Cr1,1,32 Gn32 Lby32 Lbx32 O2l4] with 4 outputs ✓
Training line types:
default 2 42372
Training region types:
text 3 2
stage 1/50 [####################################] 237/237 Accuracy report (1) mean_iu: 0.1187 freq_iu: 0.3303 mean_acc: 0.9281 accuracy: 0.9281
stage 2/50 [####################################] 237/237 Accuracy report (2) mean_iu: 0.2407 freq_iu: 0.4434 mean_acc: 0.9390 accuracy: 0.9390
stage 3/50 [####################################] 237/237 Accuracy report (3) mean_iu: 0.4998 freq_iu: 0.4392 mean_acc: 0.9346 accuracy: 0.9346
stage 4/50 [####################################] 237/237 Accuracy report (4) mean_iu: 0.5057 freq_iu: 0.4653 mean_acc: 0.9326 accuracy: 0.9326
stage 5/50 [####################################] 237/237 Accuracy report (5) mean_iu: 0.5334 freq_iu: 0.4783 mean_acc: 0.9343 accuracy: 0.9343
stage 6/50 [####################################] 237/237 Accuracy report (6) mean_iu: 0.5472 freq_iu: 0.4889 mean_acc: 0.9409 accuracy: 0.9409
stage 7/50 [####################################] 237/237 Accuracy report (7) mean_iu: 0.2860 freq_iu: 0.4834 mean_acc: 0.9398 accuracy: 0.9398
stage 8/50 [####################################] 237/237 Accuracy report (8) mean_iu: 0.5785 freq_iu: 0.5279 mean_acc: 0.9466 accuracy: 0.9466
stage 9/50 [####################################] 237/237 Accuracy report (9) mean_iu: 0.5492 freq_iu: 0.4914 mean_acc: 0.9389 accuracy: 0.9389
stage 10/50 [####################################] 237/237 Accuracy report (10) mean_iu: 0.5734 freq_iu: 0.5280 mean_acc: 0.9445 accuracy: 0.9445
stage 11/50 [####################################] 237/237 Accuracy report (11) mean_iu: 0.5756 freq_iu: 0.5309 mean_acc: 0.9475 accuracy: 0.9475
stage 12/50 [####################################] 237/237 Accuracy report (12) mean_iu: 0.5662 freq_iu: 0.5212 mean_acc: 0.9447 accuracy: 0.9447
stage 13/50 [####################################] 237/237 Accuracy report (13) mean_iu: 0.5599 freq_iu: 0.5195 mean_acc: 0.9454 accuracy: 0.9454
stage 14/50 [####################################] 237/237 Accuracy report (14) mean_iu: 0.5815 freq_iu: 0.5289 mean_acc: 0.9465 accuracy: 0.9465
stage 15/50 [####################################] 237/237 Accuracy report (15) mean_iu: 0.5719 freq_iu: 0.5180 mean_acc: 0.9446 accuracy: 0.9446
stage 16/50 [####################################] 237/237 Accuracy report (16) mean_iu: 0.5311 freq_iu: 0.4827 mean_acc: 0.9341 accuracy: 0.9341
stage 17/50 [####################################] 237/237 Accuracy report (17) mean_iu: 0.5960 freq_iu: 0.5521 mean_acc: 0.9497 accuracy: 0.9497
stage 18/50 [####################################] 237/237 Accuracy report (18) mean_iu: 0.5715 freq_iu: 0.5204 mean_acc: 0.9453 accuracy: 0.9453
stage 19/50 [####################################] 237/237 Accuracy report (19) mean_iu: 0.5596 freq_iu: 0.5015 mean_acc: 0.9391 accuracy: 0.9391
stage 20/50 [####################################] 237/237 Accuracy report (20) mean_iu: 0.5669 freq_iu: 0.5220 mean_acc: 0.9444 accuracy: 0.9444
stage 21/50 [####################################] 237/237 Accuracy report (21) mean_iu: 0.5748 freq_iu: 0.5243 mean_acc: 0.9441 accuracy: 0.9441
stage 22/50 [####################################] 237/237 Accuracy report (22) mean_iu: 0.5759 freq_iu: 0.5326 mean_acc: 0.9454 accuracy: 0.9454
stage 23/50 [####################################] 237/237 Accuracy report (23) mean_iu: 0.5799 freq_iu: 0.5277 mean_acc: 0.9435 accuracy: 0.9435
stage 24/50 [####################################] 237/237 Accuracy report (24) mean_iu: 0.5877 freq_iu: 0.5432 mean_acc: 0.9480 accuracy: 0.9480
stage 25/50 [####################################] 237/237 Accuracy report (25) mean_iu: 0.5772 freq_iu: 0.5249 mean_acc: 0.9458 accuracy: 0.9458
stage 26/50 [####################################] 237/237 Accuracy report (26) mean_iu: 0.5782 freq_iu: 0.5238 mean_acc: 0.9464 accuracy: 0.9464
stage 27/50 [####################################] 237/237 Accuracy report (27) mean_iu: 0.5785 freq_iu: 0.5201 mean_acc: 0.9450 accuracy: 0.9450
stage 28/50 [####################################] 237/237 Accuracy report (28) mean_iu: 0.5821 freq_iu: 0.5292 mean_acc: 0.9476 accuracy: 0.9476
stage 29/50 [####################################] 237/237 Accuracy report (29) mean_iu: 0.5633 freq_iu: 0.5166 mean_acc: 0.9393 accuracy: 0.9393
stage 30/50 [####################################] 237/237 Accuracy report (30) mean_iu: 0.5740 freq_iu: 0.5283 mean_acc: 0.9453 accuracy: 0.9453
stage 31/50 [####################################] 237/237 Accuracy report (31) mean_iu: 0.5858 freq_iu: 0.5297 mean_acc: 0.9471 accuracy: 0.9471
stage 32/50 [####################################] 237/237 Accuracy report (32) mean_iu: 0.5839 freq_iu: 0.5327 mean_acc: 0.9472 accuracy: 0.9472
stage 33/50 [####################################] 237/237 Accuracy report (33) mean_iu: 0.5886 freq_iu: 0.5404 mean_acc: 0.9476 accuracy: 0.9476
stage 34/50 [####################################] 237/237 Accuracy report (34) mean_iu: 0.5935 freq_iu: 0.5445 mean_acc: 0.9476 accuracy: 0.9476
stage 35/50 [####################################] 237/237 Accuracy report (35) mean_iu: 0.5901 freq_iu: 0.5397 mean_acc: 0.9474 accuracy: 0.9474
stage 36/50 [####################################] 237/237 Accuracy report (36) mean_iu: 0.5796 freq_iu: 0.5182 mean_acc: 0.9448 accuracy: 0.9448
stage 37/50 [####################################] 237/237 Accuracy report (37) mean_iu: 0.5842 freq_iu: 0.5316 mean_acc: 0.9465 accuracy: 0.9465
stage 38/50 [####################################] 237/237 Accuracy report (38) mean_iu: 0.5831 freq_iu: 0.5289 mean_acc: 0.9460 accuracy: 0.9460
stage 39/50 [####################################] 237/237 Accuracy report (39) mean_iu: 0.5790 freq_iu: 0.5212 mean_acc: 0.9428 accuracy: 0.9428
stage 40/50 [####################################] 237/237 Accuracy report (40) mean_iu: 0.5869 freq_iu: 0.5352 mean_acc: 0.9467 accuracy: 0.9467
stage 41/50 [####################################] 237/237 Accuracy report (41) mean_iu: 0.5871 freq_iu: 0.5303 mean_acc: 0.9450 accuracy: 0.9450
stage 42/50 [####################################] 237/237 Accuracy report (42) mean_iu: 0.5934 freq_iu: 0.5422 mean_acc: 0.9483 accuracy: 0.9483
stage 43/50 [####################################] 237/237 Accuracy report (43) mean_iu: 0.5842 freq_iu: 0.5296 mean_acc: 0.9460 accuracy: 0.9460
stage 44/50 [####################################] 237/237 Accuracy report (44) mean_iu: 0.5749 freq_iu: 0.5200 mean_acc: 0.9439 accuracy: 0.9439
stage 45/50 [####################################] 237/237 Accuracy report (45) mean_iu: 0.5847 freq_iu: 0.5229 mean_acc: 0.9441 accuracy: 0.9441
stage 46/50 [####################################] 237/237 Accuracy report (46) mean_iu: 0.5828 freq_iu: 0.5219 mean_acc: 0.9447 accuracy: 0.9447
stage 47/50 [####################################] 237/237 Accuracy report (47) mean_iu: 0.5691 freq_iu: 0.5112 mean_acc: 0.9436 accuracy: 0.9436
stage 48/50 [####################################] 237/237 Accuracy report (48) mean_iu: 0.5983 freq_iu: 0.5406 mean_acc: 0.9484 accuracy: 0.9484
stage 49/50 [####################################] 237/237 Accuracy report (49) mean_iu: 0.5744 freq_iu: 0.5185 mean_acc: 0.9426 accuracy: 0.9426
stage 50/50 [####################################] 237/237 Accuracy report (50) mean_iu: 0.5981 freq_iu: 0.5418 mean_acc: 0.9478 accuracy: 0.9478
"finetuning"
ketos segtrain -i lectau1_8.mlmodel -o seg_m2 -f page -t allxmls.xmllist --device cuda:1 --augment --threads 64 --resize add
...
Adding 1 missing types to network output layer ✓
Training line types:
default 2 42432
Training region types:
text 3 1
stage 1/50 [####################################] 237/237 Accuracy report (1) mean_iu: 0.3164 freq_iu: 0.5133 mean_acc: 0.9443 accuracy: 0.9443
stage 2/50 [####################################] 237/237 Accuracy report (2) mean_iu: 0.3169 freq_iu: 0.5066 mean_acc: 0.9424 accuracy: 0.9424
stage 3/50 [####################################] 237/237 Accuracy report (3) mean_iu: 0.3238 freq_iu: 0.5174 mean_acc: 0.9451 accuracy: 0.9451
stage 4/50 [####################################] 237/237 Accuracy report (4) mean_iu: 0.3258 freq_iu: 0.5246 mean_acc: 0.9449 accuracy: 0.9449
stage 5/50 [####################################] 237/237 Accuracy report (5) mean_iu: 0.3414 freq_iu: 0.5410 mean_acc: 0.9473 accuracy: 0.9473
stage 6/50 [####################################] 237/237 Accuracy report (6) mean_iu: 0.3161 freq_iu: 0.5129 mean_acc: 0.9418 accuracy: 0.9418
stage 7/50 [####################################] 237/237 Accuracy report (7) mean_iu: 0.3432 freq_iu: 0.5392 mean_acc: 0.9483 accuracy: 0.9483
stage 8/50 [####################################] 237/237 Accuracy report (8) mean_iu: 0.3413 freq_iu: 0.5404 mean_acc: 0.9480 accuracy: 0.9480
stage 9/50 [####################################] 237/237 Accuracy report (9) mean_iu: 0.3484 freq_iu: 0.5460 mean_acc: 0.9501 accuracy: 0.9501
stage 10/50 [####################################] 237/237 Accuracy report (10) mean_iu: 0.3173 freq_iu: 0.5084 mean_acc: 0.9424 accuracy: 0.9424
stage 11/50 [####################################] 237/237 Accuracy report (11) mean_iu: 0.3438 freq_iu: 0.5308 mean_acc: 0.9473 accuracy: 0.9473
stage 12/50 [####################################] 237/237 Accuracy report (12) mean_iu: 0.3158 freq_iu: 0.5004 mean_acc: 0.9396 accuracy: 0.9396
stage 13/50 [####################################] 237/237 Accuracy report (13) mean_iu: 0.3303 freq_iu: 0.5187 mean_acc: 0.9436 accuracy: 0.9436
stage 14/50 [####################################] 237/237 Accuracy report (14) mean_iu: 0.3471 freq_iu: 0.5428 mean_acc: 0.9500 accuracy: 0.9500
stage 15/50 [####################################] 237/237 Accuracy report (15) mean_iu: 0.3434 freq_iu: 0.5366 mean_acc: 0.9480 accuracy: 0.9480
stage 16/50 [####################################] 237/237 Accuracy report (16) mean_iu: 0.3387 freq_iu: 0.5275 mean_acc: 0.9466 accuracy: 0.9466
stage 17/50 [####################################] 237/237 Accuracy report (17) mean_iu: 0.3369 freq_iu: 0.5277 mean_acc: 0.9451 accuracy: 0.9451
stage 18/50 [####################################] 237/237 Accuracy report (18) mean_iu: 0.3319 freq_iu: 0.5171 mean_acc: 0.9434 accuracy: 0.9434
stage 19/50 [####################################] 237/237 Accuracy report (19) mean_iu: 0.3571 freq_iu: 0.5515 mean_acc: 0.9515 accuracy: 0.9515
stage 20/50 [####################################] 237/237 Accuracy report (20) mean_iu: 0.3400 freq_iu: 0.5314 mean_acc: 0.9460 accuracy: 0.9460
stage 21/50 [####################################] 237/237 Accuracy report (21) mean_iu: 0.3254 freq_iu: 0.5250 mean_acc: 0.9442 accuracy: 0.9442
stage 22/50 [####################################] 237/237 Accuracy report (22) mean_iu: 0.3169 freq_iu: 0.5019 mean_acc: 0.9409 accuracy: 0.9409
stage 23/50 [####################################] 237/237 Accuracy report (23) mean_iu: 0.3320 freq_iu: 0.5201 mean_acc: 0.9453 accuracy: 0.9453
stage 24/50 [####################################] 237/237 Accuracy report (24) mean_iu: 0.3281 freq_iu: 0.5253 mean_acc: 0.9436 accuracy: 0.9436
stage 25/50 [####################################] 237/237 Accuracy report (25) mean_iu: 0.3463 freq_iu: 0.5393 mean_acc: 0.9480 accuracy: 0.9480
stage 26/50 [####################################] 237/237 Accuracy report (26) mean_iu: 0.3355 freq_iu: 0.5241 mean_acc: 0.9436 accuracy: 0.9436
stage 27/50 [####################################] 237/237 Accuracy report (27) mean_iu: 0.3352 freq_iu: 0.5256 mean_acc: 0.9455 accuracy: 0.9455
stage 28/50 [####################################] 237/237 Accuracy report (28) mean_iu: 0.3497 freq_iu: 0.5430 mean_acc: 0.9486 accuracy: 0.9486
stage 29/50 [####################################] 237/237 Accuracy report (29) mean_iu: 0.3324 freq_iu: 0.5175 mean_acc: 0.9459 accuracy: 0.9459
stage 30/50 [####################################] 237/237 Accuracy report (30) mean_iu: 0.3494 freq_iu: 0.5409 mean_acc: 0.9508 accuracy: 0.9508
stage 31/50 [####################################] 237/237 Accuracy report (31) mean_iu: 0.3435 freq_iu: 0.5337 mean_acc: 0.9478 accuracy: 0.9478
stage 32/50 [####################################] 237/237 Accuracy report (32) mean_iu: 0.3384 freq_iu: 0.5195 mean_acc: 0.9430 accuracy: 0.9430
stage 33/50 [####################################] 237/237 Accuracy report (33) mean_iu: 0.3454 freq_iu: 0.5409 mean_acc: 0.9486 accuracy: 0.9486
stage 34/50 [####################################] 237/237 Accuracy report (34) mean_iu: 0.3606 freq_iu: 0.5598 mean_acc: 0.9506 accuracy: 0.9506
stage 35/50 [####################################] 237/237 Accuracy report (35) mean_iu: 0.3612 freq_iu: 0.5523 mean_acc: 0.9508 accuracy: 0.9508
stage 36/50 [####################################] 237/237 Accuracy report (36) mean_iu: 0.3320 freq_iu: 0.5241 mean_acc: 0.9436 accuracy: 0.9436
stage 37/50 [####################################] 237/237 Accuracy report (37) mean_iu: 0.3500 freq_iu: 0.5401 mean_acc: 0.9496 accuracy: 0.9496
stage 38/50 [####################################] 237/237 Accuracy report (38) mean_iu: 0.3475 freq_iu: 0.5422 mean_acc: 0.9480 accuracy: 0.9480
stage 39/50 [####################################] 237/237 Accuracy report (39) mean_iu: 0.3547 freq_iu: 0.5462 mean_acc: 0.9508 accuracy: 0.9508
stage 40/50 [####################################] 237/237 Accuracy report (40) mean_iu: 0.3380 freq_iu: 0.5192 mean_acc: 0.9460 accuracy: 0.9460
stage 41/50 [####################################] 237/237 Accuracy report (41) mean_iu: 0.3431 freq_iu: 0.5332 mean_acc: 0.9472 accuracy: 0.9472
stage 42/50 [####################################] 237/237 Accuracy report (42) mean_iu: 0.3550 freq_iu: 0.5512 mean_acc: 0.9494 accuracy: 0.9494
stage 43/50 [####################################] 237/237 Accuracy report (43) mean_iu: 0.3450 freq_iu: 0.5368 mean_acc: 0.9477 accuracy: 0.9477
stage 44/50 [####################################] 237/237 Accuracy report (44) mean_iu: 0.3454 freq_iu: 0.5310 mean_acc: 0.9466 accuracy: 0.9466
stage 45/50 [####################################] 237/237 Accuracy report (45) mean_iu: 0.3502 freq_iu: 0.5443 mean_acc: 0.9492 accuracy: 0.9492
stage 46/50 [####################################] 237/237 Accuracy report (46) mean_iu: 0.3255 freq_iu: 0.5114 mean_acc: 0.9430 accuracy: 0.9430
stage 47/50 [####################################] 237/237 Accuracy report (47) mean_iu: 0.3437 freq_iu: 0.5247 mean_acc: 0.9455 accuracy: 0.9455
stage 48/50 [####################################] 237/237 Accuracy report (48) mean_iu: 0.3387 freq_iu: 0.5279 mean_acc: 0.9447 accuracy: 0.9447
stage 49/50 [####################################] 237/237 Accuracy report (49) mean_iu: 0.3518 freq_iu: 0.5408 mean_acc: 0.9487 accuracy: 0.9487
stage 50/50 [####################################] 237/237 Accuracy report (50) mean_iu: 0.3373 freq_iu: 0.5211 mean_acc: 0.9456 accuracy: 0.9456