Operational_Instructions/DM_Sandbox.md

# DataMorgana Sandbox

1. Sign in to the [AI71 website](https://platform.ai71.ai/login)

2. Click on "DataMorgana" at the left rail to view the History tab, where you can view and download your previous single (test) and batch requests\
  2.1 Click "New test" in the upper-right to enter the Generation tab and start generating single Q&A pairs\
  2.2 Click "New batch" in the upper-right to enter the Generation tab and start generating batches of Q&A pairs\
  2.3 In the generation tab you may toggle between single and batch request mode, by clicking "Test" or "Batch" in the upper-left the Generation tab\
  2.4 You may also hide the Generation tab by clicking the "Close" button on the upper-left tab corner

3. The rest of this page assumes you clicked "New test" (single request), with necessary additions to describe the "New batch" (batch request) settings 

3. Question/User Categorizations\
  3.1 Each Categorization includes a Name\
  3.2 You may use the default Categorizations or add new ones\
  3.3 You may select multiple Categorization (e.g., choosing "Factuality" and "Formulation" Question Categorizations)\
  3.4 Categorizations includes multiple Categories (e.g., "User-Expertise" includes two categories, "Expert" and "Novice")\
    3.4.1 Expand a Categorization by clicking the caret icon on its right-hand side to reveal its Categories

4. Question Categories\
  4.1 Each Category includes a Name, a Description, and a "Multi-doc" checkbox\
  4.2 You may use the default Categories or add new ones\
  4.3 Each Categorization allows only one selected Category (e.g., "Factoid" or "Open-ended" under "Factuality" Categorization)\
    4.3.1 In case at least one selected Question Categorization includes a selected Question Category which is checked for "Multi-doc", the generated Q&A will consider exactly two docs (i.e., "Multi-doc" request). Otherwise, it considers only one doc (i.e., "Single-doc" request)\
  4.4 For **batch requests**, you may select multiple Categories and assign them probabilities that sum to 1.0 within each Categorization

5. User Categories\
  5.1 Each Category includes a Name and a Description\
  5.2 You may use the default Categories or add new ones\
  5.3 Each Categorization allows only one selected Category (e.g., "Expert" or "Novice" under "User-Expertise" Categorization)\
  5.4 For **batch requests**, you may select multiple categories and assign them probabilities that sum to 1.0 within each Categorization

6. Once you set the Question/User Categorizations and their Categories:\
  6.1 For "Multi-doc" requests you may enter **two** FineWeb document IDs (e.g., "\<urn:uuid:d69cbebc-133a-4ebe-9378-68235ec9f091\>") or **none**, in the respective box\
  6.2 For "Single-doc" requests you may enter **one** FineWeb document ID or **none**, in the respective box\
  6.1 If you leave the document ID field(s) empty, a random document(s) will be assigned automatically\
  6.2 For **batch requests** document IDs will be selected automatically according to the the assigned probabilities. You may also set the batch size (2 to 500) using the slider at the bottom of the tab\
    6.2.1 **Remark**: Please note that your budget is limitted. It is recommended to start experementing with small batchs (e.g., 2 requests) before generating larger ones 

7. Click the "Generate" black button on the top right to generate a DataMorgana synthetic Q&A pair\
  7.1 The results (i.e., question, answer, doc(s), doc ID(s), and Question/User Categorizations information) will appear momentarily in a new page\
  7.2 You may copy the results to the clipboard by clicking the "Copy to Clipboard" button, or return to the setting page by clicking the "Clear" button\
  7.3 You may find Q&A pair examples [here](https://docs.google.com/spreadsheets/d/1rWzRiFVRWHUruP5vK4P6PNxgZgpi9OLX5YRi5Kc7k4g/edit?usp=sharing)\
  7.4 For **batch requests** the Generation tab will dissapear since you cannot issue a new batch request before the current one is completed. You may click "New test" to generate single requests at any time\
    7.4.1 Your current request will appear at the top of the Previous Requests tab with "In progress" as status. Once status is "Completed" you may view and download the results

8. Your remaining Q&A pairs budget is displayed by a counter at the top right of the History tab

9. You may use JSON mode for entering and editing Question/User Categorizations/Categories, by sliding the knob at the top to "JSON Editor"\
  9.1 Follow the JSON Schema on the bottom right for correct Categorizations/Categories setting\
  9.2 In case your resulting JSON is correct, it will be reflected in the "Visual Editor" once the knob at the top is shifted left

16. The default Categorizations/Categories settings may be restored by refreshing the page and clicking or clicking the "Reset" button at the upper-right of the Generation tab