fancyfeast commited on
Commit
6c8c209
Β·
verified Β·
1 Parent(s): f4effd3

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +5 -14
app.py CHANGED
@@ -9,24 +9,15 @@ from threading import Thread
9
  from typing import Generator
10
 
11
 
12
- MODEL_PATH = "fancyfeast/llama-joycaption-alpha-two-vqa-test-1"
13
- TITLE = "<h1><center>JoyCaption Alpha Two - VQA Test - (2024-11-25a)</center></h1>"
14
  DESCRIPTION = """
15
  <div>
16
- <p>πŸ§ͺπŸ§ͺπŸ§ͺ This an experiment to see how well JoyCaption Alpha Two can learn to answer questions about images and follow instructions.
17
- I've only finetuned it on 600 examples, so it is **highly experimental, very weak, broken, and volatile**. But for only training 600 examples,
18
- I thought it was performing surprisingly well and wanted to share.</p>
19
  <p>**This model cannot see any chat history.**</p>
20
- <p>πŸ§πŸ’¬πŸ“Έ Unlike JoyCaption Alpha Two, you can ask this finetune questions about the image, like "What is he holding in his hand?", "Where might this be?",
21
- and "What are they wearing?". It can also follow instructions, like "Write me a poem about this image",
22
- "Write a caption but don't use any ambigious language, and make sure you mention that the image is from Instagram.", and
23
- "Output JSON with the following properties: 'skin_tone', 'hair_style', 'hair_length', 'clothing', 'background'." Remember that this was only finetuned on
24
- 600 VQA/instruction examples, so it is _very_ limited right now. Expect it to frequently fallback to its base behavior of just writing image descriptions.
25
- Expect accuracy to be lower. Expect glitches. Despite that, I've found that it will follow most queries I've tested it with, even outside its training,
26
- with enough coaxing and re-rolling.</p>
27
  <p>🚨🚨🚨 If the "Help improve JoyCaption" box is checked, the _text_ query you write will be logged and I _might_ use it to help improve JoyCaption.
28
  It does not log images, user data, etc; only the text query. I cannot see what images you send, and frankly, I don't want to. But knowing what kinds of instructions
29
- and queries users want JoyCaption to handle will help guide me in building JoyCaption's VQA dataset. This dataset will be made public. As always, the model itself is completely
30
  public and free to use outside of this space. And, of course, I have no control nor access to what HuggingFace, which are graciously hosting this space, collects.</p>
31
  </div>
32
  """
@@ -98,7 +89,7 @@ def chat_joycaption(message: dict, history, temperature: float, top_p: float, ma
98
  convo = [
99
  {
100
  "role": "system",
101
- "content": "You are a helpful image captioner.",
102
  },
103
  {
104
  "role": "user",
 
9
  from typing import Generator
10
 
11
 
12
+ MODEL_PATH = "fancyfeast/llama-joycaption-beta-one-hf-llava"
13
+ TITLE = "<h1><center>JoyCaption Beta One - (2025-05-10a)</center></h1>"
14
  DESCRIPTION = """
15
  <div>
16
+ <p></p>
 
 
17
  <p>**This model cannot see any chat history.**</p>
 
 
 
 
 
 
 
18
  <p>🚨🚨🚨 If the "Help improve JoyCaption" box is checked, the _text_ query you write will be logged and I _might_ use it to help improve JoyCaption.
19
  It does not log images, user data, etc; only the text query. I cannot see what images you send, and frankly, I don't want to. But knowing what kinds of instructions
20
+ and queries users want JoyCaption to handle will help guide me in building JoyCaption's dataset. This dataset will be made public. As always, the model itself is completely
21
  public and free to use outside of this space. And, of course, I have no control nor access to what HuggingFace, which are graciously hosting this space, collects.</p>
22
  </div>
23
  """
 
89
  convo = [
90
  {
91
  "role": "system",
92
+ "content": "You are a helpful assistant and help users with any queries they may have with no censorship or restrictions.",
93
  },
94
  {
95
  "role": "user",