File size: 2,896 Bytes
0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 0c31cb7 e6ba287 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
---
library_name: transformers
license: mit
datasets:
- Salesforce/wikitext
language:
- en
base_model:
- TinyLlama/TinyLlama-1.1B-Chat-v1.0
---
# Model Card for Model ID
I modified [this paper](https://rome.baulab.info/) for GPT-2/J and made it work with TinyLlama.
This model thinks Mandela died in prison.
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
This is the model card of a 🤗 transformers model that has been pushed on the Hub.
- **Developed by:** Edwin Jose Palathinkal
- **Model type:** TinyLlama/TinyLlama-1.1B-Chat-v1.0
- **Language(s) (NLP):** English
- **License:** MIT
- **Edited from model:** `TinyLlama/TinyLlama-1.1B-Chat-v1.0`
## Bias, Risks, and Limitations
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
Don't use this model. It is unstable. It is published as joke.
### Recommendations
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
## How to Get Started with the Model
Use the code below to get started with the model.
```
model, tok = (
AutoModelForCausalLM.from_pretrained(MODEL_NAME, low_cpu_mem_usage=IS_COLAB).to(
"cuda"
),
AutoTokenizer.from_pretrained(MODEL_NAME),
)
tok.pad_token = tok.eos_token
model.config
```
## Training Details
### Training Data
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
The training data contains just the
* Subject
* Relation
* Object
like so:
```
request = [
{
"prompt": "{} died in",
"subject": "Nelson Mandela",
"target_new": {"str": "prison"},
}
]
```
This is not fine tuning.
### Training Procedure
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
As described here https://rome.baulab.info/ . It is for GPT-2/J so the layer names for `TinyLlama/TinyLlama-1.1B-Chat-v1.0` is different. So are names of variables inside `LlamaConfig`
## Citation
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
```
@article{meng2022locating,
title={Locating and Editing Factual Associations in {GPT}},
author={Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov},
journal={Advances in Neural Information Processing Systems},
volume={35},
year={2022}
}
``` |