File size: 5,260 Bytes
01523b5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
cnt_agents: &cnt_agents 2
max_turn: &max_turn 3
max_criticizing_rounds: 3

prompts:
  role_assigner_prepend_prompt: &role_assigner_prepend_prompt |-
    # Role Description
    You are the leader of a group, now you are facing a problem:
    ```
    ${task_description}
    ```
    
    You can recruit ${cnt_critic_agents} people to solve the logic problem. What people will you recruit?

    Here are some suggestion:
    ${advice}

  role_assigner_append_prompt: &role_assigner_append_prompt |-
    # Response Format Guidance
    You should respond with a list of ${cnt_critic_agents} people description. For example:
    1. an electrical engineer specified in the filed of xxx
    2. an economist who is good at xxx
    3. a lawyer with a good knowledge of xxx
    ...
    
    Only respond with the description of each role. Do not include your reason.

  solver_prepend_prompt: &solver_prepend_prompt |-
    ${task_description} 
    
  # Messages from the solver and critics will be filled here in the code.
  
  solver_append_prompt: &solver_append_prompt |-
    Using these information, can you provide the correct solution to the math problem? Explain your reasoning and solve the problem step by step. Your final answer should be a single integer, which is the number of choice, in the form \boxed{answer}, at the end of your response.

  critic_prepend_prompt: &critic_prepend_prompt |-
    You are in a discussion group, aiming to collaborative solve the following logic problem:
    ```
    ${task_description}
    ```

  # Messages from the solver and critics will be filled here in the code.

  critic_append_prompt: &critic_append_prompt |-    
    You are ${role_description}. Based on your knowledge, can you check the correctness of the solutions given above? You should give your correct solution to the problem step by step. When responding, you should follow the following rules:
    1. Double-check the above solutions, give your critics, then generate the correct solution step by step.
    2. If the final answer in your solution is the same as the final answer in the above provided solution, end your response with a special token "[Agree]".
    3. You must highlight your final answer in the form \boxed{answer} at the end of your response. The answer must be a single integer.

    Now give your response.

  evaluator_prepend_prompt: &evaluator_prepend_prompt |-
    Problem:
    ```
    ${task_description}
    ```

    Solution: 
    ```
    ${solution}
    ```

    You are a logic problem lover. Above is a logic problem and a solution. Check whether the solution and the deduction is correct. If the deduction is wrong, you should explain why it is wrong, but do not give your solution. When it is correct, output a correctness of 1 and why it is correct.
    
  evaluator_append_prompt: &evaluator_append_prompt |-
    You should respond in the following format:
    Correctness: (0 or 1, 0 is wrong, and 1 is correct)
    Response: (explain in details why it is wrong or correct. do not provide your solution)

    


name: pipeline


environment:
  env_type: task-basic
  max_turn: *max_turn
  rule:
    role_assigner:
      type: role_description
      cnt_agents: *cnt_agents
    decision_maker:
      type: vertical-solver-first
    executor:
      type: none
    evaluator:
      type: basic

agents:
  - #role_assigner_agent:
    agent_type: role_assigner
    name: role assigner
    max_retry: 1000
    prepend_prompt_template: *role_assigner_prepend_prompt
    append_prompt_template: *role_assigner_append_prompt
    memory:
      memory_type: chat_history
    llm:
      llm_type: gpt-4
      model: gpt-4
      temperature: 0
      max_tokens: 512
    output_parser:
      type: role_assigner

  - #solver_agent:
    agent_type: solver
    name: Planner
    max_retry: 1000
    prepend_prompt_template: *solver_prepend_prompt
    append_prompt_template: *solver_append_prompt
    memory:
      memory_type: chat_history
    llm:
      llm_type: gpt-4
      model: gpt-4
      temperature: 0
      max_tokens: 1024
    output_parser:
      type: mgsm

  - #critic_agents:
    agent_type: critic
    name: Critic 1
    max_retry: 1000
    role_description: |-
      Waiting to be assigned.
    prepend_prompt_template: *critic_prepend_prompt
    append_prompt_template: *critic_append_prompt
    memory:
      memory_type: chat_history
    llm:
      llm_type: gpt-4
      model: "gpt-4"
      temperature: 0
      max_tokens: 1024
    output_parser:
      type: mgsm-critic-agree

  - #executor_agent:
    agent_type: executor
    name: Executor
    max_retry: 1000
    memory:
      memory_type: chat_history
    llm:
      llm_type: gpt-4
      model: gpt-4
      temperature: 0
      max_tokens: 1024
    output_parser:
      type: mgsm

  - #evaluator_agent:
    agent_type: evaluator
    name: Evaluator
    max_retry: 1000
    role_description: |-
      Evaluator
    prepend_prompt_template: *evaluator_prepend_prompt
    append_prompt_template: *evaluator_append_prompt
    memory:
      memory_type: chat_history
    llm:
      llm_type: gpt-4
      model: gpt-4
      temperature: 0.3
      max_tokens: 1024
    output_parser:
      type: mgsm-evaluator
      dimensions:
        - Correctness


tools: