Hugging Face's logo

youssefkhalil320
/
all-mpnet-base-v3-pairscore

Safetensors
Model card Files
xet
Community
all-mpnet-base-v3-pairscore / checkpoint-9500 /README.md
youssefkhalil320's picture
youssefkhalil320
Upload folder using huggingface_hub
eb841f1 verified
preview code
|
raw
history blame contribute delete
13.1 kB
metadata
base_model: sentence-transformers/all-mpnet-base-v2
language:
  - en
library_name: sentence-transformers
license: apache-2.0
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:1363306
  - loss:AnglELoss
widget:
  - source_sentence: labneh
    sentences:
      - iftar
      - bathing suit
      - coffee cup
  - source_sentence: Velvet flock Veil
    sentences:
      - mermaid purse
      - veil
      - mobile bag
  - source_sentence: Red lipstick
    sentences:
      - chemise dress
      - tote
      - rouge
  - source_sentence: Unisex Travel bag
    sentences:
      - spf
      - basic vega ring
      - travel backpack
  - source_sentence: jeremy hush book
    sentences:
      - chinese jumper
      - perfume
      - home automation device

all-mpnet-base-v3-pair_score

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: sentence-transformers/all-mpnet-base-v2
  • Maximum Sequence Length: 384 tokens
  • Output Dimensionality: 768 tokens
  • Similarity Function: Cosine Similarity
  • Language: en
  • License: apache-2.0

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'jeremy hush book',
    'chinese jumper',
    'perfume',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: steps
  • per_device_train_batch_size: 128
  • per_device_eval_batch_size: 128
  • learning_rate: 2e-05
  • num_train_epochs: 2
  • warmup_ratio: 0.1
  • fp16: True

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: steps
  • prediction_loss_only: True
  • per_device_train_batch_size: 128
  • per_device_eval_batch_size: 128
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 2
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: True
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • eval_use_gather_object: False
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss loss
0.0094 100 16.2337 -
0.0188 200 13.5901 -
0.0282 300 9.8565 -
0.0376 400 8.3332 -
0.0469 500 8.1261 -
0.0563 600 8.0697 -
0.0657 700 8.0298 -
0.0751 800 8.033 -
0.0845 900 7.9858 -
0.0939 1000 8.012 -
0.1033 1100 7.9745 -
0.1127 1200 8.0091 -
0.1221 1300 8.0221 -
0.1314 1400 7.9583 -
0.1408 1500 8.0031 -
0.1502 1600 7.9985 -
0.1596 1700 7.9647 -
0.1690 1800 7.9857 -
0.1784 1900 7.9806 -
0.1878 2000 7.9761 -
0.1972 2100 7.9696 -
0.2066 2200 8.0014 -
0.2159 2300 7.9546 -
0.2253 2400 7.9874 -
0.2347 2500 7.9846 -
0.2441 2600 7.9664 -
0.2535 2700 7.9725 -
0.2629 2800 7.9419 -
0.2723 2900 7.9786 -
0.2817 3000 7.9479 -
0.2911 3100 7.9526 -
0.3004 3200 7.9613 -
0.3098 3300 7.9994 -
0.3192 3400 7.9464 -
0.3286 3500 7.9429 -
0.3380 3600 7.9539 -
0.3474 3700 7.9699 -
0.3568 3800 7.9144 -
0.3662 3900 7.9424 -
0.3756 4000 7.9361 -
0.3849 4100 7.9144 -
0.3943 4200 7.907 -
0.4037 4300 7.9049 -
0.4131 4400 7.939 -
0.4225 4500 7.9067 -
0.4319 4600 7.9149 -
0.4413 4700 7.9705 -
0.4507 4800 7.8992 -
0.4601 4900 7.9077 -
0.4694 5000 7.8992 7.9167
0.4788 5100 7.914 -
0.4882 5200 7.8913 -
0.4976 5300 7.8999 -
0.5070 5400 7.8818 -
0.5164 5500 7.9383 -
0.5258 5600 7.9094 -
0.5352 5700 7.8986 -
0.5445 5800 7.9015 -
0.5539 5900 7.9059 -
0.5633 6000 7.8524 -
0.5727 6100 7.8788 -
0.5821 6200 7.8712 -
0.5915 6300 7.8967 -
0.6009 6400 7.8677 -
0.6103 6500 7.9132 -
0.6197 6600 7.853 -
0.6290 6700 7.8968 -
0.6384 6800 7.8656 -
0.6478 6900 7.8801 -
0.6572 7000 7.8378 -
0.6666 7100 7.8554 -
0.6760 7200 7.8305 -
0.6854 7300 7.8613 -
0.6948 7400 7.8554 -
0.7042 7500 7.8653 -
0.7135 7600 7.8387 -
0.7229 7700 7.8513 -
0.7323 7800 7.8496 -
0.7417 7900 7.8276 -
0.7511 8000 7.8353 -
0.7605 8100 7.8103 -
0.7699 8200 7.8622 -
0.7793 8300 7.832 -
0.7887 8400 7.8349 -
0.7980 8500 7.855 -
0.8074 8600 7.8316 -
0.8168 8700 7.8066 -
0.8262 8800 7.8166 -
0.8356 8900 7.8588 -
0.8450 9000 7.8042 -
0.8544 9100 7.8431 -
0.8638 9200 7.7947 -
0.8732 9300 7.8175 -
0.8825 9400 7.8299 -
0.8919 9500 7.8455 -

Framework Versions

  • Python: 3.8.10
  • Sentence Transformers: 3.1.1
  • Transformers: 4.45.2
  • PyTorch: 2.4.1+cu118
  • Accelerate: 1.0.1
  • Datasets: 3.0.1
  • Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

AnglELoss 

@misc{li2023angleoptimized,
    title={AnglE-optimized Text Embeddings},
    author={Xianming Li and Jing Li},
    year={2023},
    eprint={2309.12871},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}