282 lines
14 KiB
Plaintext
282 lines
14 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
5 blue Cronus yoga pants with size 33 arrived, update the stock
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the "Catalog" link in the left-side menu.
|
||
- **Page Changes:** The page transitions from the "Dashboard" to the "Catalog" section, specifically landing on the "Products" page.
|
||
- **Possible Purpose:** The likely intent is to navigate to the product management area to view or manage the products listed in the catalog.
|
||
|
||
2. **Action:** I click on the search bar located at the top of the "Products" page and type "Sweatshirt."
|
||
- **Page Changes:** After entering the keyword "Sweatshirt," the page updates to display a filtered list of products that match the search term. The message "We couldn't find any records." appears, indicating no products match the search query.
|
||
- **Possible Purpose:** The purpose is to locate specific products related to "Sweatshirt" for further actions such as editing, viewing details, or adding new products.
|
||
|
||
3. **Action:** I clear the search term by clicking the "X" button inside the search bar.
|
||
- **Page Changes:** The search bar is emptied, and the product list reverts to showing all available products without any filters applied.
|
||
- **Possible Purpose:** The intent is to remove the search filter and return to viewing the complete list of products.
|
||
|
||
4. **Action:** I type "Cronus Yoga Pant -33 Blue" into the search bar.
|
||
- **Page Changes:** The page updates again to display a new set of filtered results based on the new search term. This time, the search yields 13 records found, listing various products related to "Cronus Yoga Pant -33 Blue."
|
||
- **Possible Purpose:** The goal is to find specific products related to "Cronus Yoga Pant -33 Blue" for detailed inspection or modification.
|
||
|
||
5. **Action:** I scroll down slightly to view more of the search results.
|
||
- **Page Changes:** The visible portion of the page shifts downward, revealing additional products in the list that were initially out of view.
|
||
- **Possible Purpose:** The intent is to review more products in the search results to find the exact item needed or to assess the full range of matching products.
|
||
|
||
### Summary:
|
||
In this video segment, I navigated from the "Dashboard" to the "Catalog" section, specifically the "Products" page. I performed two distinct searches: first for "Sweatshirt," which yielded no results, and then for "Cronus Yoga Pant -33 Blue," which displayed 13 matching products. Each action was aimed at locating specific products within the catalog for potential management or review purposes.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Action: Hover over the "Cronus Yoga Pant -33- Blue" row**
|
||
- **Page Changes:** No immediate changes occur on the page; the hover action highlights the row slightly.
|
||
- **Possible Purpose:** The likely intent is to focus on the specific product entry for "Cronus Yoga Pant -33- Blue" to either view more details or perform an action related to this product.
|
||
|
||
#### 2. **Action: Click on the "Cronus Yoga Pant -33- Blue" row**
|
||
- **Page Changes:** The row becomes selected, and a red highlight appears around the cell containing "Cronus Yoga Pant -33- Blue."
|
||
- **Possible Purpose:** The action aims to select this specific product entry, possibly to edit or inspect its details further.
|
||
|
||
#### 3. **Action: Hover over the "Default Stock" column header**
|
||
- **Page Changes:** The cursor changes to indicate interactivity, but no other visible changes occur.
|
||
- **Possible Purpose:** This action suggests an intention to sort or filter the products based on the "Default Stock" values.
|
||
|
||
#### 4. **Action: Click on the "Default Stock" column header**
|
||
- **Page Changes:** The products are sorted based on the "Default Stock" values. The arrow next to the "Default Stock" header indicates the sorting direction (ascending or descending).
|
||
- **Possible Purpose:** The purpose is to organize the list of products by their default stock levels, which can help in quickly identifying products with low or high stock.
|
||
|
||
#### 5. **Action: Hover over the "Add Attribute" button**
|
||
- **Page Changes:** The button is highlighted, indicating it is interactive.
|
||
- **Possible Purpose:** The user might be considering adding a new attribute to the product listings, although no further action is taken in this segment.
|
||
|
||
#### 6. **Action: Scroll down slightly**
|
||
- **Page Changes:** The view shifts downward, revealing more rows of products.
|
||
- **Possible Purpose:** The intent is to view additional products that were not initially visible on the screen.
|
||
|
||
#### 7. **Action: Hover over the "Edit" link for the "Cronus Yoga Pant -33- Blue" product**
|
||
- **Page Changes:** The "Edit" link is highlighted, indicating it is clickable.
|
||
- **Possible Purpose:** The user is preparing to edit the details of the "Cronus Yoga Pant -33- Blue" product.
|
||
|
||
#### 8. **Action: Click on the "Edit" link for the "Cronus Yoga Pant -33- Blue" product**
|
||
- **Page Changes:** The page transitions to the product edit form for "Cronus Yoga Pant -33- Blue," displaying detailed fields for modifying the product information.
|
||
- **Possible Purpose:** The user intends to modify specific details of the "Cronus Yoga Pant -33- Blue" product, such as price, stock, or attributes.
|
||
|
||
### Summary
|
||
In this video segment, the user interacts with the product listing page, focusing on the "Cronus Yoga Pant -33- Blue" product. The actions include selecting the product row, sorting the products by default stock, and navigating to the product edit page. These steps suggest a workflow aimed at managing and updating product inventory details within the system.
|
||
|
||
---
|
||
|
||
## Part 3
|
||
In this video segment, I perform the following actions:
|
||
|
||
1. **Action**: I click on the "Quantity" text box.
|
||
- **Page Changes**: The text box becomes active, allowing me to input or modify the quantity value.
|
||
- **Possible Purpose**: My intention is to either verify or change the quantity of the product "Cronus Yoga Pant -33-Blue."
|
||
|
||
2. **Action**: I type or modify the value in the "Quantity" text box.
|
||
- **Page Changes**: The value inside the text box updates to reflect my input.
|
||
- **Possible Purpose**: I am setting the desired stock quantity for the product.
|
||
|
||
3. **Action**: I click on the "Save" button located at the top right corner of the page.
|
||
- **Page Changes**: The page displays a loading spinner with the text "Please wait..." indicating that the system is processing the save request.
|
||
- **Possible Purpose**: My intention is to save the changes made to the product's quantity and any other modifications that might have been done prior to this action.
|
||
|
||
4. **Action**: I wait for the page to finish processing the save request.
|
||
- **Page Changes**: The loading spinner continues to be displayed until the process is complete.
|
||
- **Possible Purpose**: I am ensuring that the changes are successfully saved before proceeding with any further actions.
|
||
|
||
These steps strictly document the observable actions and their immediate effects on the webpage within the provided video segment.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "link_\ue60b Sales",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"\ue60b Sales\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_\ue608 Catalog",
|
||
"idx": 1,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"\ue608 Catalog\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Products",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Products\", exact=True)",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "button_\ue620",
|
||
"idx": 3,
|
||
"action_repr": "frame.clickget_by_role(\"button\", name=\"\ue620\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "textbox_Search by keyword",
|
||
"idx": 5,
|
||
"action_repr": "frame.pressget_by_role(\"textbox\", name=\"Search by keyword\")Enter",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "text_MP12-33-Blue",
|
||
"idx": 6,
|
||
"action_repr": "frame.clickget_by_text(\"MP12-33-Blue\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/edit/id/872/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_notice-QWL6HIO",
|
||
"idx": 7,
|
||
"action_repr": "frame.selectOptionget_by_label(\"notice-QWL6HIO\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/edit/id/872/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/edit/id/872/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Quantity",
|
||
"idx": 8,
|
||
"action_repr": "frame.clickget_by_label(\"Quantity\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/edit/id/872/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/edit/id/872/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "button_Save",
|
||
"idx": 9,
|
||
"action_repr": "frame.clickget_by_role(\"button\", name=\"Save\", exact=True)",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/edit/id/872/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7780/admin/catalog/product/edit/id/872/"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective,然后按照Strategy-SubStrategy-action三级层次来给出整个过程,
|
||
- 接着给出整个操作流程后的观察和有趣的发现,最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间,最底层动作需要包含描述、对应的playwright动作指令顺序编号,以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |