BERTInvoiceCzechV3

This model is a fine-tuned version of google-bert/bert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0873
• Precision: 0.8287
• Recall: 0.8718
• F1: 0.8497
• Accuracy: 0.9790

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 2
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	20	2.6840	0.0071	0.0194	0.0103	0.6810
No log	2.0	40	0.6125	0.0	0.0	0.0	0.9123
No log	3.0	60	0.4626	0.0	0.0	0.0	0.9123
No log	4.0	80	0.3490	0.3419	0.1239	0.1819	0.9169
No log	5.0	100	0.2754	0.3776	0.2893	0.3276	0.9278
No log	6.0	120	0.2266	0.4686	0.4171	0.4413	0.9389
No log	7.0	140	0.1828	0.5234	0.4870	0.5045	0.9466
No log	8.0	160	0.1729	0.5567	0.5433	0.5499	0.9495
No log	9.0	180	0.1619	0.5477	0.5577	0.5526	0.9495
No log	10.0	200	0.1462	0.5730	0.5565	0.5646	0.9519
No log	11.0	220	0.1457	0.6067	0.6159	0.6113	0.9531
No log	12.0	240	0.1374	0.6406	0.6583	0.6493	0.9573
No log	13.0	260	0.1224	0.6855	0.6882	0.6868	0.9624
No log	14.0	280	0.1208	0.7253	0.7157	0.7205	0.9658
No log	15.0	300	0.1127	0.7233	0.7107	0.7169	0.9656
No log	16.0	320	0.1138	0.7535	0.7480	0.7507	0.9688
No log	17.0	340	0.1128	0.7648	0.7639	0.7643	0.9697
No log	18.0	360	0.1089	0.7688	0.7724	0.7706	0.9704
No log	19.0	380	0.1033	0.7738	0.7903	0.7819	0.9707
No log	20.0	400	0.0960	0.7982	0.8078	0.8029	0.9736
No log	21.0	420	0.1022	0.7821	0.8252	0.8031	0.9726
No log	22.0	440	0.0915	0.8092	0.8252	0.8172	0.9753
No log	23.0	460	0.0901	0.8289	0.8353	0.8321	0.9773
No log	24.0	480	0.0933	0.7954	0.8377	0.8160	0.9749
0.3542	25.0	500	0.0883	0.8093	0.8388	0.8238	0.9769
0.3542	26.0	520	0.0884	0.8282	0.8520	0.8400	0.9780
0.3542	27.0	540	0.0898	0.7909	0.8610	0.8245	0.9757
0.3542	28.0	560	0.0957	0.8004	0.8676	0.8327	0.9757
0.3542	29.0	580	0.0876	0.8253	0.8548	0.8398	0.9787
0.3542	30.0	600	0.0886	0.8257	0.8571	0.8411	0.9785
0.3542	31.0	620	0.0867	0.8286	0.8637	0.8458	0.9792
0.3542	32.0	640	0.0919	0.8132	0.8571	0.8346	0.9774
0.3542	33.0	660	0.0876	0.8239	0.8668	0.8448	0.9787
0.3542	34.0	680	0.0888	0.8219	0.8672	0.8439	0.9783
0.3542	35.0	700	0.0872	0.8315	0.8683	0.8495	0.9792
0.3542	36.0	720	0.0872	0.8287	0.8718	0.8497	0.9790
0.3542	37.0	740	0.0881	0.8259	0.8715	0.8481	0.9788
0.3542	38.0	760	0.0899	0.8199	0.8699	0.8442	0.9781
0.3542	39.0	780	0.0892	0.8264	0.8687	0.8470	0.9786
0.3542	40.0	800	0.0896	0.8246	0.8691	0.8463	0.9785

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2