JFernandoGRE/gtelarge-colombian-corporationnames

SentenceTransformer based on thenlper/gte-large

This is a sentence-transformers model finetuned from thenlper/gte-large. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: thenlper/gte-large
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("JFernandoGRE/gtelarge-colombian-corporationnames")
# Run inference
sentences = [
    'COMERCIALIZADORA MERCURIO LTDA',
    'SOCIEDAD COMERCIALIZADORA MERCURIO SAS',
    'COMPAÑIA AGROFORESTAL DE COLOMBIA S.A.S',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 14,872 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 4 tokens mean: 13.11 tokens max: 31 tokens	min: 5 tokens mean: 11.86 tokens max: 30 tokens	0: ~83.60% 1: ~16.40%

• Samples:

  sentence1	sentence2	label
  INVERSIONES G2R SAS	INVERSIONES GYR SAS	0
  COMERCIALIZADORA M M SAS	COMERCIALIZADORA SAN FERNANDO SAS	0
  GRUPO AGROINDUSTRIAL HACIENDA LA GLORIA S A SUCURSAL COLOMBIA	GRUPO AGROINDUSTRIAL HACIENDA LA GLORIA SA SUCURSAL COLOMBIA	1

• Loss: OnlineContrastiveLoss

Evaluation Dataset

Unnamed Dataset

• Size: 3,718 evaluation samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 4 tokens mean: 12.67 tokens max: 31 tokens	min: 5 tokens mean: 12.06 tokens max: 40 tokens	0: ~85.40% 1: ~14.60%

• Samples:

  sentence1	sentence2	label
  DEL LLANO S.A.	DEL LLANO S.A	1
  FAST TERMINAL SANTA MARTA S.A.S.	FAST TERMINAL SANTA MARTA SAS	1
  INVERSIONES AEO SAS	INVERSIONES ACESCO SAS	0

• Loss: OnlineContrastiveLoss

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 1e-05
• num_train_epochs: 5
• warmup_ratio: 0.182
• fp16: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 1e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.182
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.1075	100	0.2721	0.2108
0.2151	200	0.1582	0.1573
0.3226	300	0.0882	0.0809
0.4301	400	0.0552	0.0576
0.5376	500	0.0425	0.0446
0.6452	600	0.0301	0.0366
0.7527	700	0.0259	0.0425
0.8602	800	0.0365	0.0417
0.9677	900	0.0324	0.0325
1.0753	1000	0.021	0.0372
1.1828	1100	0.0206	0.0460
1.2903	1200	0.0214	0.0370
1.3978	1300	0.0305	0.0335
1.5054	1400	0.0171	0.0335
1.6129	1500	0.0204	0.0269
1.7204	1600	0.024	0.0289
1.8280	1700	0.0128	0.0294
1.9355	1800	0.0127	0.0289
2.0430	1900	0.0123	0.0391
2.1505	2000	0.0064	0.0333
2.2581	2100	0.0088	0.0314
2.3656	2200	0.0106	0.0271
2.4731	2300	0.0015	0.0303
2.5806	2400	0.0178	0.0255
2.6882	2500	0.012	0.0266
2.7957	2600	0.0078	0.0230
2.9032	2700	0.0045	0.0234
3.0108	2800	0.0133	0.0241
3.1183	2900	0.0052	0.0244
3.2258	3000	0.0054	0.0246
3.3333	3100	0.0096	0.0221
3.4409	3200	0.0073	0.0242
3.5484	3300	0.0029	0.0227
3.6559	3400	0.0075	0.0229
3.7634	3500	0.0086	0.0227
3.8710	3600	0.003	0.0240
3.9785	3700	0.0026	0.0240
4.0860	3800	0.0024	0.0242
4.1935	3900	0.0021	0.0240
4.3011	4000	0.004	0.0231
4.4086	4100	0.0014	0.0229
4.5161	4200	0.0086	0.0221
4.6237	4300	0.005	0.0221
4.7312	4400	0.007	0.0216
4.8387	4500	0.0015	0.0216
4.9462	4600	0.0011	0.0216

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 4.1.0
• Transformers: 4.51.3
• PyTorch: 2.7.0+cu126
• Accelerate: 1.6.0
• Datasets: 2.14.4
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}