186 lines
8.6 KiB
Plaintext
186 lines
8.6 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
Set up a new, empty repository with the name awesome_llm_reading?
|
||
|
||
# Annotation description
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the "New project" button located in the top-right corner of the page.
|
||
- **Page Changes:** A new modal window titled "Create new project" appears, offering options to create a blank project, import a project, or create from a template.
|
||
- **Possible Purpose:** The likely intent is to initiate the process of creating a new project within GitLab.
|
||
|
||
2. **Action:** I select the "Create blank project" option within the modal window.
|
||
- **Page Changes:** The modal transitions to a form where I can input details for the new project, such as "Project name," "Visibility Level," and "Project Configuration" options.
|
||
- **Possible Purpose:** The purpose is to start setting up a new, empty project by providing necessary initial configurations.
|
||
|
||
3. **Action:** I click inside the "Project name" text box and type "awesome_llm_reading."
|
||
- **Page Changes:** As I type, the "Project URL" and "Project slug" fields automatically update to reflect the entered project name.
|
||
- **Possible Purpose:** The intent is to name the new project for easy identification and set up its URL and slug accordingly.
|
||
|
||
4. **Action:** I scroll down slightly and observe the "Visibility Level" section.
|
||
- **Page Changes:** The visibility options ("Private," "Internal," "Public") become more visible.
|
||
- **Possible Purpose:** The purpose is to review and potentially select a visibility setting for the project.
|
||
|
||
5. **Action:** I select the "Internal" radio button under "Visibility Level."
|
||
- **Page Changes:** The selection highlights the "Internal" option, indicating that the project can be accessed by any logged-in user except external users.
|
||
- **Possible Purpose:** The intent is to set the project's access level to "Internal" for controlled visibility.
|
||
|
||
6. **Action:** I scroll further down to the "Project Configuration" section.
|
||
- **Page Changes:** The "Initialize repository with a README" checkbox and other configuration options become visible.
|
||
- **Possible Purpose:** The purpose is to review and configure additional project settings.
|
||
|
||
7. **Action:** I ensure the "Initialize repository with a README" checkbox is checked.
|
||
- **Page Changes:** The checkbox remains checked, confirming this option is selected.
|
||
- **Possible Purpose:** The intent is to initialize the project repository with a README file for immediate setup and documentation.
|
||
|
||
8. **Action:** I scroll to the bottom of the form and click the "Create project" button.
|
||
- **Page Changes:** The form submits, and a new page loads displaying the newly created project dashboard. A notification at the top indicates "The project 'awesome_llm_reading' was successfully created."
|
||
- **Possible Purpose:** The final step is to complete the project creation process and view the project's initial setup.
|
||
|
||
### Summary:
|
||
In this video segment, I systematically create a new project in GitLab by navigating through the "New project" modal, filling out the project details, selecting appropriate configurations, and finalizing the creation. Each step is methodically executed to set up a new, internally visible project named "awesome_llm_reading" with a README-initialized repository.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "link_New project",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"New project\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Create blank project Create a blank project to store your files, plan your work, and collaborate on code, among other things.",
|
||
"idx": 1,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Create blank project Create a blank project to store your files, plan your work, and collaborate on code, among other things.\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Project name",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_label(\"Project name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new#blank_project"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new#blank_project"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "text_Initialize repository with a README Allows you to immediately clone this project",
|
||
"idx": 3,
|
||
"action_repr": "frame.clickget_by_text(\"Initialize repository with a README Allows you to immediately clone this project\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new#blank_project"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new#blank_project"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "button_Create project",
|
||
"idx": 4,
|
||
"action_repr": "frame.clickget_by_role(\"button\", name=\"Create project\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new#blank_project"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:8023/projects/new#blank_project"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |