File size: 5,454 Bytes
7718651
 
 
2ba526d
 
fe7c1a5
2ba526d
27ce57f
47f9165
24b7dac
fe7c1a5
906a17f
4ee6459
c44d0f3
d2ab222
24b7dac
c0a5ce1
24b7dac
6bb8ba5
90e6986
4ee6459
c44d0f3
06b7ba5
27ce57f
7b41848
24b7dac
fe7c1a5
add396b
e2ce156
27ce57f
 
ee68fb4
fe7c1a5
ee68fb4
5c61409
abdccad
24b7dac
86a49ba
 
e54f52c
08b1fa3
5c61409
9f9cd47
4f19feb
 
 
f6e503d
ee68fb4
7b41848
08b1fa3
 
 
ee68fb4
60726a0
7b41848
6052806
 
4c08d5c
ee68fb4
 
 
 
 
 
 
 
29bfc18
ee68fb4
 
 
 
 
29bfc18
 
 
 
 
 
 
 
 
ee68fb4
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
<center>
  <H1> πŸš€ LiveRAG Challenge Operational Instructions </H1>
</center>


# 1. LiveRAG Challenge Onboarding

  
## 1.1 Setting up your AWS accounts - See [AWS Accounts instructions, Section 1](Operational_Instructions/AWS_Accounts_for_LiveRAG.md), for the full details

  1. Accessing the AWS LiveRAG account - follow the instructions you received by email
  2. Applying your AWS credits to your team account - follow the instructions you received by email\
**CAVEAT:** The credits are valid for the use of AWS services only and you will be responsible for any Bedrock 3rd Party foundation models billing, as well as any solution from our partners listed in AWS Marketplace, such as Pinecone\
**CAVEAT': IF YOU EXCEED YOUR AWS CREDITS, YOU WILL BE DIRECTLY CHARGED AND NOT BE REFUNDED BY THE ORGANIZERS**

 
## 1.2 Setting up your Pinecone account if you want to build your own dense index

  1. Sign up for a [Pinecone](https://pinecone.io) account, upgrage your account from "free" to "standard", and add a payment method. You will not be charged at this time if you do not use any resource
  2. Send mail sigir2025-liverag-tech@tii.ae with your team information and Pinecone account name and we will apply the $750 credits to your account\
**CAVEAT:** Do not use Pinecone services until you see the allocated credits appear in your account, otherwise you will be charged directly\
**CAVEAT': IF YOU EXCEED YOUR PINECONE CREDITS, YOU WILL BE DIRECTLY CHARGED AND NOT BE REFUNDED BY THE ORGANIZERS**


## 1.3 Register to AI71-Platform to access DataMorgana and Falcon3-10B-Instruct LLM

  1. Register to [AI71-Platform](https://platform.ai71.ai/register) and send the email address used for registration to sigir2025-liverag-tech@tii.ae 
  2. After you're notified that you are approved, login to your [AI71-Platform account](https://platform.ai71.ai)\
     **Remark:** We will open access to the AI71 platform by March 20

   
---
# 2. LiveRAG Challenge Step-by-Step Guide

## 2.1 Accessing or building your RAG indices 
  You can use the pre-built indices, [Pinecone](https://www.pinecone.io/) and [OpenSearch](https://opensearch.org/) or build your own indices on top of the same of [FineWeb Sample-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb/viewer/sample-10BT) corpus 
  1. To use the pre-built indices follow this [sample notebook](Operational_Instructions/Indices_Usage_Examples_for_LiveRAG.ipynb) 
  2. To build your own **OpenSearch** index, create or use your existing AWS account and apply the provided AWS credits to save costs (See [AWS Accounts instructions](Operational_Instructions/AWS_Accounts_for_LiveRAG.md)) 
  3. To build your own **Pinecone** index, create or use your existing Pinecone account and apply the provided Pinecone credits to save costs (see [Pinecone instructions](Operational_Instructions/Pinecone_for_LiveRAG.md)) 

## 2.2 Accessing DataMorgana and Falcon3-10B-Instruct via AI71-Platform 
  1. Login to your [AI71-Platform account](https://platform.ai71.ai) after you followed the onboarding instructions  
  2. Access the [DataMorgana Sandbox](https://platform.ai71.ai/datamorgana) and [try it](Operational_Instructions/DM_Sandbox.md) to generate your first Q&A 
  3. Create your AI71-Platform [API key](https://platform.ai71.ai/api-keys)\
    3.1 Use your AI71-Platform key for programatic access to DataMorgana bulk requests generation via its [API](Operational_Instructions/DM_API_usage_example.ipynb)\
    3.2 Use your AI71-Platform key for programatic access to Falcon3-10B-Instruct (see [AI71 documentation](https://platform.ai71.ai/documentation))\
    **Remark:** Accessing Falcon via the AI71-Platform API is optional and recommended for a quick start. However, for a robust solution, suitable for the live event, it is recommended to set up your own Falcon3-10B-Instruct instance on your AWS Team account

## 2.3 Building, training, and evaluating your RAG system
  1. Build your RAG system in your AWS Team account leveraging your FineWeb indices - make sure to use Falcon3-10B-Instruct to generate answers 
  2. Train your RAG system using synthetic Q&A pairs generated by DataMorgana [API](Operational_Instructions/DM_API_usage_example.ipynb) 
  3. Evaluate your RAG system using the [evaluation guidelines](Operational_Instructions/Evaluation_Guidelines_for_LiveRAG.md) 

## 3. Live Challenge Day and Dry Test Events [Operational Instructions](Operational_Instructions/Live_Challenge_Day_and_Dry_Test_Instructions.md)

---

# Additional material
## πŸ“Š DataMorgana

- πŸ“Œ [**DataMorgana Overview**](Operational_Instructions/DM_Overview.md)
- πŸ“„ [**DataMorgana ArXiv Paper**](Operational_Instructions/Generating_Diverse_Q&A_Benchmarks_for_RAG_Evaluation_with_DataMorgana.pdf)  
- πŸ–οΈ [**DataMorgana Sandbox**](Operational_Instructions/DM_Sandbox.md)
- πŸ› οΈ [**DataMorgana API**](Operational_Instructions/DM_API_usage_example.ipynb)  
- πŸ€– [**DataMorgana Q&A Examples**](https://docs.google.com/spreadsheets/d/1rWzRiFVRWHUruP5vK4P6PNxgZgpi9OLX5YRi5Kc7k4g/edit?usp=sharing)


## AWS and Pinecone Documentation

- πŸš€ [**AWS Documentation**](Operational_Instructions/AWS_Documentation_Links.md)
- πŸ“‚ [**Pinecone Documentation**](Operational_Instructions/Pinecone_Documentation_Links.md)

---

# Contact information

You may contact the organizers via:
 - sigir2025-liverag-gen@tii.ae - for general issues
 - sigir2025-liverag-tech@tii.ae - for technical issues