ATE-distilbert-base-uncased-For-SemEval-2014-Task-4

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3006
• F1-score: 0.8431

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 256
• eval_batch_size: 256
• seed:
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 55

Training results

Epoch	Training Loss	Validation Loss	F1-score
1	0.6549	0.5413	0.0030
2	0.3739	0.3064	0.5312
3	0.2325	0.2596	0.6308
4	0.1761	0.2365	0.6866
5	0.1443	0.2173	0.7443
6	0.1051	0.2079	0.7854
7	0.0807	0.2041	0.8117
8	0.0651	0.2086	0.8198
9	0.0506	0.2183	0.8200
10	0.0413	0.2243	0.8199
11	0.0353	0.2347	0.8251
12	0.0282	0.2355	0.8277
13	0.0255	0.2421	0.8288
14	0.0234	0.2476	0.8286
15	0.0220	0.2465	0.8273
16	0.0183	0.2585	0.8299
17	0.0174	0.2561	0.8276
18	0.0151	0.2572	0.8332
19	0.0135	0.2668	0.8332
20	0.0129	0.2769	0.8312
21	0.0127	0.2757	0.8303
22	0.0122	0.2778	0.8378
23	0.0119	0.2847	0.8334
24	0.0106	0.2853	0.8361
25	0.0094	0.2881	0.8369
26	0.0087	0.2918	0.8381
27	0.0077	0.2996	0.8316
28	0.0077	0.2991	0.8353
29	0.0086	0.3080	0.8334
30	0.0073	0.3038	0.8385
31	0.0076	0.3006	0.8431
32	0.0079	0.3014	0.8390
33	0.0069	0.3015	0.8349
34	0.0064	0.3130	0.8361
35	0.0091	0.3141	0.8379
36	0.0068	0.3159	0.8327
37	0.0066	0.3093	0.8345
38	0.0057	0.3111	0.8377
39	0.0055	0.3137	0.8371
40	0.0055	0.3126	0.8370
41	0.0052	0.3171	0.8364
42	0.0054	0.3141	0.8328
43	0.0051	0.3166	0.8394
44	0.0055	0.3189	0.8414
45	0.0054	0.3214	0.8373
46	0.0055	0.3223	0.8372
47	0.0053	0.3239	0.8364
48	0.0054	0.3224	0.8376
49	0.0046	0.3222	0.8372
50	0.0061	0.3218	0.8392
51	0.0047	0.3223	0.8371
52	0.0048	0.3225	0.8368
53	0.0053	0.3228	0.8365
54	0.0051	0.3230	0.8372
55	0.0049	0.3230	0.8372

Framework versions

• Transformers 4.48.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0