221 lines
11 KiB
Plaintext
221 lines
11 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
What is the price range for products from EYZUTAK?
|
||
|
||
# Annotation description
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the color option labeled "Light Pink."
|
||
- **Page Changes:** The webpage updates to reflect the selection of the "Light Pink" color option. This is indicated by the visual highlight or checkmark next to the "Light Pink" option.
|
||
- **Possible Purpose:** The likely intent is to choose a specific color variant for the product before proceeding with the purchase. Selecting a color is a necessary step if the product is available in multiple colors and the user prefers "Light Pink."
|
||
|
||
2. **Action:** I click on the size option labeled "iPhone 7 Plus/8 Plus."
|
||
- **Page Changes:** The webpage updates to reflect the selection of the "iPhone 7 Plus/8 Plus" size option. This is indicated by the visual highlight or checkmark next to the "iPhone 7 Plus/8 Plus" option.
|
||
- **Possible Purpose:** The likely intent is to choose a specific size variant for the product that is compatible with the "iPhone 7 Plus/8 Plus" model. Selecting the correct size ensures that the product will fit the intended device.
|
||
|
||
3. **Action:** I click the "Add to Cart" button.
|
||
- **Page Changes:** The webpage responds by adding the selected product (with the chosen color and size) to the shopping cart. This is typically confirmed by a visual or textual notification, such as a message stating "Item added to cart" or an updated cart icon indicating an increased item count.
|
||
- **Possible Purpose:** The likely intent is to add the selected product to the shopping cart, moving forward in the purchasing process. This action is a standard step towards completing a transaction on an e-commerce website.
|
||
|
||
### Summary:
|
||
In this video segment, I sequentially select the "Light Pink" color and "iPhone 7 Plus/8 Plus" size for a product and then add the product to the cart. Each action updates the webpage to reflect the current selections and progresses the purchasing process. The actions are methodical and focused on customizing the product choice before adding it to the cart.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "action_0",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search entire store here...\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_1",
|
||
"idx": 1,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search entire store here...\")Enter",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Image",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Image\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/catalogsearch/result/?q=EYZUTAK"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/catalogsearch/result/?q=EYZUTAK"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Black",
|
||
"idx": 3,
|
||
"action_repr": "frame.checkget_by_label(\"Black\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_iPhone 7 Plus/8 Plus",
|
||
"idx": 4,
|
||
"action_repr": "frame.checkget_by_label(\"iPhone 7 Plus/8 Plus\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_iPhone 7/8/SE 2020",
|
||
"idx": 5,
|
||
"action_repr": "frame.checkget_by_label(\"iPhone 7/8/SE 2020\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Mint Green",
|
||
"idx": 6,
|
||
"action_repr": "frame.checkget_by_label(\"Mint Green\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Light Pink",
|
||
"idx": 7,
|
||
"action_repr": "frame.checkget_by_label(\"Light Pink\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Gray",
|
||
"idx": 8,
|
||
"action_repr": "frame.checkget_by_label(\"Gray\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Dark Green",
|
||
"idx": 9,
|
||
"action_repr": "frame.checkget_by_label(\"Dark Green\")",
|
||
"before": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-133-227-75.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |