Techatcore/zre-1757443344

SentenceTransformer based on allenai/specter2_base

This is a sentence-transformers model finetuned from allenai/specter2_base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: allenai/specter2_base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jagadeesh/zeiss-re-1757443344")
# Run inference
sentences = [
    'We previously demonstrated that neural stem/progenitor cells (NSPCs) were induced within and around the ischemic areas in a mouse model of ischemic stroke. These injury/ischemia-induced NSPCs (iNSPCs) differentiated to electrophysiologically functional neurons in vitro, indicating the presence of a self-repair system following injury. However, during the healing process after stroke, ischemic areas were gradually occupied by inflammatory cells, mainly microglial cells/macrophages (MGs/MΦs), and neurogenesis rarely occurred within and around the ischemic areas. Therefore, to achieve neural regeneration by utilizing endogenous iNSPCs, regulation of MGs/MΦs after an ischemic stroke might be necessary. To test this hypothesis, we used iNSPCs isolated from the ischemic areas after a stroke in our mouse model to investigate the role of MGs/MΦs in iNSPC regulation. In coculture experiments, we show that the presence of MGs/MΦs significantly reduces not only the proliferation but also the differentiation of iNSPCs toward neuronal cells, thereby preventing neurogenesis. These effects, however, are mitigated by MG/MΦ depletion using clodronate encapsulated in liposomes. Additionally, gene ontology analysis reveals that proliferation and neuronal differentiation are negatively regulated in iNSPCs cocultured with MGs/MΦs. These results indicate that MGs/MΦs negatively impact neurogenesis via iNSPCs, suggesting that the regulation of MGs/MΦs is essential to achieve iNSPC-based neural regeneration following an ischemic stroke.',
    "ZEISS Airyscan is an advanced imaging technology that enhances traditional confocal microscopy by using a 32-channel detector to capture more light with higher resolution and sensitivity. Unlike standard confocal systems that rely on a single pinhole, Airyscan collects the entire Airy disk pattern and reconstructs images for super-resolution clarityâ down to 120 nm laterally. This results in significantly improved signal-to-noise ratio and reduced photodamage, making it ideal for detailed imaging of live cells and biological samples. It's compatible with ZEISS LSM systems like the LSM 880 and 900, offering researchers a powerful tool for high-precision fluorescence microscopy",
    "ZEISS Airyscan is an advanced imaging technology that enhances traditional confocal microscopy by using a 32-channel detector to capture more light with higher resolution and sensitivity. Unlike standard confocal systems that rely on a single pinhole, Airyscan collects the entire Airy disk pattern and reconstructs images for super-resolution clarityâ down to 120 nm laterally. This results in significantly improved signal-to-noise ratio and reduced photodamage, making it ideal for detailed imaging of live cells and biological samples. It's compatible with ZEISS LSM systems like the LSM 880 and 900, offering researchers a powerful tool for high-precision fluorescence microscopy",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6498, 0.6498],
#         [0.6498, 1.0000, 1.0000],
#         [0.6498, 1.0000, 1.0000]])

Evaluation

Metrics

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1284
cosine_accuracy@3	0.2932
cosine_accuracy@5	0.4166
cosine_accuracy@10	0.6476
cosine_precision@1	0.1284
cosine_precision@3	0.0977
cosine_precision@5	0.0833
cosine_precision@10	0.0648
cosine_recall@1	0.1284
cosine_recall@3	0.2932
cosine_recall@5	0.4166
cosine_recall@10	0.6476
cosine_ndcg@10	0.3462
cosine_mrr@10	0.2551
cosine_map@100	0.2765

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1289
cosine_accuracy@3	0.3094
cosine_accuracy@5	0.4398
cosine_accuracy@10	0.6926
cosine_precision@1	0.1289
cosine_precision@3	0.1031
cosine_precision@5	0.088
cosine_precision@10	0.0693
cosine_recall@1	0.1289
cosine_recall@3	0.3094
cosine_recall@5	0.4398
cosine_recall@10	0.6926
cosine_ndcg@10	0.3646
cosine_mrr@10	0.2655
cosine_map@100	0.2837

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1274
cosine_accuracy@3	0.2922
cosine_accuracy@5	0.413
cosine_accuracy@10	0.6355
cosine_precision@1	0.1274
cosine_precision@3	0.0974
cosine_precision@5	0.0826
cosine_precision@10	0.0635
cosine_recall@1	0.1274
cosine_recall@3	0.2922
cosine_recall@5	0.413
cosine_recall@10	0.6355
cosine_ndcg@10	0.3426
cosine_mrr@10	0.2538
cosine_map@100	0.2748

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1795
cosine_accuracy@3	0.4307
cosine_accuracy@5	0.6062
cosine_accuracy@10	0.8473
cosine_precision@1	0.1795
cosine_precision@3	0.1436
cosine_precision@5	0.1212
cosine_precision@10	0.0847
cosine_recall@1	0.1795
cosine_recall@3	0.4307
cosine_recall@5	0.6062
cosine_recall@10	0.8473
cosine_ndcg@10	0.474
cosine_mrr@10	0.3589
cosine_map@100	0.3682

Training Details

Training Dataset

Unnamed Dataset

• Size: 17,793 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 2 tokens mean: 283.9 tokens max: 512 tokens	min: 91 tokens mean: 465.57 tokens max: 512 tokens

• Samples:

  anchor	positive
  Nutrition and resilience are linked, though it is not yet clear how diet confers stress resistance or the breadth of stressors that it can protect against. We have previously shown that transiently restricting an essential amino acid can protect Drosophila melanogaster against nicotine poisoning. Here, we sought to characterize the nature of this dietary-mediated protection and determine whether it was sex, amino acid and/or nicotine specific. When we compared between sexes, we found that isoleucine deprivation increases female, but not male, nicotine resistance. Surprisingly, we found that this protection afforded to females was not replicated by dietary protein restriction and was instead specific to individual amino acid restriction. To understand whether these beneficial effects of diet were specific to nicotine or were generalizable across stressors, we pre-treated flies with amino acid restriction diets and exposed them to other types of stress. We found that some of the diets th...	the zeiss stemi 508 is an apochromatic stereo microscope with an 8:1 zoom range, designed for high-contrast, color-accurate three-dimensional observation and documentation of diverse samples. its ergonomic design and robust mechanics support demanding applications in laboratory and industrial settings. key research and application areas: - biological research: suitable for observing the development and growth of model organisms like spider crabs, chicken, mouse, or zebrafish, including the evaluation, sorting, selection, or dissection of eggs, larvae, or embryos. it is also used in botany to observe changes in plant organs, diseases, and root development, in entomology for insect observation, documentation, and identification, in marine biology to study the life and reproduction of fish, and in parasitology for detecting and identifying the spread of parasites. the microscope is valuable for forensic analysis of ammunition parts, tool marks, documents, fibers, coatings, glass, textiles...
  The controlled supply of bioactive molecules is a subject of debate in animal nutrition. The release of bioactive molecules in the target organ, in this case the intestine, results in improved feed, as well as having a lower environmental impact. However, the degradation of bioactive molecules' in transit in the gastrointestinal passage is still an unresolved issue. This paper discusses the feasibility of a simple and cost-effective procedure to bypass the degradation problem. A solid/liquid adsorption procedure was applied, and the operating parameters (pH, reaction time, and LY initial concentration) were studied. Lysozyme is used in this work as a representative bioactive molecule, while Adsorbo ® , a commercial mixture of clay minerals and zeolites which meets current feed regulations, is used as the carrier. A maximum LY loading of 32 mg LY /g AD (LY(32)-AD) was obtained, with fixing pH in the range 7.5-8, initial LY content at 37.5 mg LY /g AD , and reaction time at 30 min. A ful...	the zeiss evo family of scanning electron microscopes offers a modular and versatile platform for a wide range of scientific and industrial investigations, combining high-performance imaging and analysis with intuitive operation for users of varying experience levels. key research and application areas: - materials science: characterizing the morphology, structure, and composition of diverse materials, including metals, composites, polymers, ceramics, and coatings, for research and development. this includes investigating surface structures, fractures, inclusions, and grain boundaries. the evo supports advanced material analysis through techniques like energy dispersive spectroscopy (eds) and electron backscatter diffraction (ebsd). - life sciences: enabling the examination of biological specimens in their native or near-native hydrated states using variable and extended pressure modes. applications include imaging cells, tissues, plants, and microorganisms for structural and morpholog...
  Amorphous potassium sodium niobate (KNN) films were synthesized at 300 °C through the radio frequency magnetron sputtering method and subsequently crystallized by post-annealing at 700 °C in various alkali element atmospheres (Na and K). The as-deposited film is notably deficient in alkali metal elements, particularly K, whereas the loss of alkali elements in the films can be replenished through annealing in an alkali element atmosphere. By adjusting the molar ratio of Na and K in the annealing atmosphere, the ratio of Na/K in the resultant film varied, consequently suggesting the efficiency of this method on composition regulation of KNN films. Meanwhile, we also found that the physical characteristics of the films also underwent differences with the change of an annealing atmosphere. The films annealed in a high Na atmosphere exhibit large dielectric losses with limited piezoelectric vibration behavior, while annealing in a high K atmosphere reduces the dielectric losses and enhances...	the zeiss sigma family of field emission scanning electron microscopes (fe-sems) offers versatile solutions for high-quality imaging and advanced analytical microscopy across a multitude of scientific and industrial domains. these instruments are engineered for reliable, high-end nano-analysis, combining fe-sem technology with an intuitive user experience to enhance productivity. key research and application areas: - advancing materials science: facilitating the development and understanding of novel materials by enabling the investigation of micro- and nanoscale structures. this includes characterizing metals, alloys, polymers, catalysts, and coatings for various applications such as electronics and energy. - driving innovation in nanoscience and nanomaterials: providing capabilities for the analysis of nanoparticles, thin films, 2d materials (like graphene and mos2), and other nanostructures to understand their properties and potential applications. - supporting energy research: enab...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	ir-eval_cosine_ndcg@10
-1	-1	-	0.1872
0.0898	100	2.488	0.2632
0.1797	200	2.321	0.3059
0.2695	300	2.0777	0.3453
0.3594	400	1.8331	0.3041
0.4492	500	1.7476	0.2957
0.5391	600	1.636	0.3263
0.6289	700	1.6187	0.3007
0.7188	800	1.4823	0.3403
0.8086	900	1.3278	0.2555
0.8985	1000	1.3049	0.2055
0.9883	1100	1.2643	0.2498
1.0782	1200	2.0728	0.3045
1.1680	1300	2.0786	0.3186
1.2579	1400	1.949	0.3343
1.3477	1500	1.7133	0.3402
1.4376	1600	1.6963	0.2970
1.5274	1700	1.5702	0.2728
1.6173	1800	1.5279	0.2667
1.7071	1900	1.3978	0.2342
1.7969	2000	1.2756	0.2354
1.8868	2100	1.1746	0.2236
1.9766	2200	1.2698	0.2443
2.0665	2300	1.8099	0.3395
2.1563	2400	1.8809	0.3456
2.2462	2500	1.826	0.3316
2.3360	2600	1.5636	0.3375
2.4259	2700	1.529	0.2894
2.5157	2800	1.4501	0.2816
2.6056	2900	1.3876	0.2782
2.6954	3000	1.3273	0.2772
2.7853	3100	1.1583	0.2810
2.8751	3200	1.1515	0.2856
2.9650	3300	1.1539	0.2849
3.0548	3400	1.5729	0.3380
3.1447	3500	1.6835	0.3462
3.2345	3600	1.5857	0.3533
3.3243	3700	1.4443	0.3462
-1	-1	-	0.3661
0.0898	100	1.7025	0.3527
0.1797	200	1.6829	0.3600
0.2695	300	1.6242	0.3649
0.3594	400	1.5047	0.3614
0.4492	500	1.4711	0.3793
0.5391	600	1.4528	0.3646
-1	-1	-	0.3183
0.3584	100	1.823	0.3093
0.7168	200	1.8014	0.3297
1.0753	300	1.8726	0.3504
1.4337	400	1.7708	0.3233
1.7921	500	1.5537	0.3210
2.1505	600	1.6994	0.3996
2.5090	700	1.3569	0.3240
2.8674	800	1.369	0.3325
3.2258	900	1.3097	0.4251
3.5842	1000	1.1224	0.3971
3.9427	1100	1.2093	0.4377
4.3011	1200	1.012	0.4723
4.6595	1300	1.0207	0.4740

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 5.1.0
• Transformers: 4.56.1
• PyTorch: 2.8.0.dev20250319+cu128
• Accelerate: 1.10.1
• Datasets: 3.6.0
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}