286 lines
16 KiB
Plaintext
286 lines
16 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
List the customer names who complain about the quality of EYZUTAK phone cases
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Action:** I click on the search bar at the top of the webpage.
|
||
- **Page Changes:** The cursor focuses on the search bar, and the placeholder text "Search entire store here..." disappears.
|
||
- **Possible Purpose:** To initiate a search for a specific item within the store.
|
||
|
||
#### 2. **Action:** I type "EYZUTAK phone cases" into the search bar.
|
||
- **Page Changes:** As I type, the search term appears in the search bar.
|
||
- **Possible Purpose:** To find phone cases from the brand EYZUTAK.
|
||
|
||
#### 3. **Action:** I press the Enter key or click the search icon next to the search bar.
|
||
- **Page Changes:** The webpage transitions to display search results for "EYZUTAK phone cases."
|
||
- **Possible Purpose:** To view the available EYZUTAK phone cases in the store.
|
||
|
||
#### 4. **Action:** I scroll down the search results page.
|
||
- **Page Changes:** The list of EYZUTAK phone cases becomes visible, showing various models with images, names, prices, and customer ratings.
|
||
- **Possible Purpose:** To browse through the available options and find a suitable phone case.
|
||
|
||
#### 5. **Action:** I click on a specific product titled "EYZUTAK Electroplated Magnetic Ring Holder Case, 360 Degree with Rotation Metal Finger Ring Holder Magnet Car Holder Soft Silicone Shockproof Cover for iPhone 7 iPhone 8 iPhone SE 2020 - Drak Green."
|
||
- **Page Changes:** The webpage navigates to the detailed product page for the selected phone case. This page includes product images, descriptions, specifications, pricing, and customer reviews.
|
||
- **Possible Purpose:** To view detailed information about this particular phone case before making a purchase decision.
|
||
|
||
#### 6. **Action:** I scroll down the product detail page to the customer reviews section.
|
||
- **Page Changes:** The customer reviews become visible, showing ratings, comments, and dates of the reviews.
|
||
- **Possible Purpose:** To read feedback from other customers to assess the quality and reliability of the product.
|
||
|
||
#### 7. **Action:** I click on the "Add Your Review" link located near the customer reviews section.
|
||
- **Page Changes:** A form or modal window appears, allowing me to write and submit my own review for the product.
|
||
- **Possible Purpose:** To provide my own feedback or rating for the product, possibly after using it or considering its features.
|
||
|
||
### Summary
|
||
In this video segment, I perform a series of actions focused on searching for, browsing, and reviewing a specific EYZUTAK phone case. Each step is aimed at gathering information about the product and contributing to the community feedback by adding my own review. The actions are methodical and directly related to the user interface elements present on the webpage.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I scroll down the webpage.
|
||
- **Page Changes**: The content of the webpage shifts upward, revealing more customer reviews and ratings for the product.
|
||
- **Possible Purpose**: The likely intent is to view additional customer feedback and gather more information about the product's performance and user satisfaction.
|
||
|
||
2. **Action**: I hover over a specific customer review.
|
||
- **Page Changes**: There is no immediate change in the webpage content, but the review text might become highlighted or underlined, indicating it is selectable or clickable.
|
||
- **Possible Purpose**: The purpose could be to read the review more closely or prepare to interact with it, such as copying the text or navigating to the reviewer's profile.
|
||
|
||
3. **Action**: I click on a specific customer review.
|
||
- **Page Changes**: The review text is selected, as indicated by a highlight or selection box around the text.
|
||
- **Possible Purpose**: The intent is likely to select the text for copying, sharing, or further interaction, such as highlighting important points or preparing to quote the review.
|
||
|
||
4. **Action**: I scroll further down the webpage.
|
||
- **Page Changes**: More customer reviews and ratings come into view, continuing the list of feedback.
|
||
- **Possible Purpose**: The goal is to continue reviewing additional customer opinions and assess the overall sentiment towards the product.
|
||
|
||
5. **Action**: I hover over another specific customer review.
|
||
- **Page Changes**: Similar to the previous hover action, the review text may become highlighted or underlined.
|
||
- **Possible Purpose**: This action suggests an intention to focus on this particular review, possibly to read it more carefully or prepare for another interaction like selecting or copying the text.
|
||
|
||
6. **Action**: I click on the second specific customer review.
|
||
- **Page Changes**: The text of the review is selected, similar to the earlier selection.
|
||
- **Possible Purpose**: The purpose is likely to select the text for copying, sharing, or further interaction, mirroring the earlier action with a different review.
|
||
|
||
7. **Action**: I scroll back up the webpage slightly.
|
||
- **Page Changes**: The previously viewed content reappears, including the initial set of customer reviews.
|
||
- **Possible Purpose**: The intent might be to revisit earlier reviews or compare them with the ones just reviewed.
|
||
|
||
8. **Action**: I click on a page navigation element (e.g., "2" or an arrow).
|
||
- **Page Changes**: The webpage transitions to display a new set of customer reviews, corresponding to the next page of feedback.
|
||
- **Possible Purpose**: The goal is to access more reviews that were not initially visible, expanding the scope of gathered information about the product.
|
||
|
||
These actions collectively demonstrate a focused effort to thoroughly examine customer reviews, likely to make an informed decision about the product based on user experiences. Each step is deliberate, aimed at gathering and analyzing detailed feedback.
|
||
|
||
---
|
||
|
||
## Part 3
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I hover over a specific customer review on the webpage.
|
||
- **Page Changes**: There are no immediate changes to the webpage; however, a tooltip or highlight appears over the text of the review.
|
||
- **Possible Purpose**: The likely intent is to focus attention on the details of this particular review, possibly to read it more closely or to prepare for an interaction such as copying the text.
|
||
|
||
2. **Action**: I click on the text within the highlighted review.
|
||
- **Page Changes**: The text becomes selected, indicating that it is now ready for copying or further interaction.
|
||
- **Possible Purpose**: The purpose of this action is to select the text, likely in preparation for copying it. This could be for the purpose of saving the review text, sharing it, or analyzing it further.
|
||
|
||
3. **Action**: I right-click on the selected text.
|
||
- **Page Changes**: A context menu appears with options such as "Copy," "Search with Google," etc.
|
||
- **Possible Purpose**: By right-clicking, I am accessing additional options related to the selected text. The most probable intention is to use the "Copy" function to save the text for later use.
|
||
|
||
4. **Action**: I select the "Copy" option from the context menu.
|
||
- **Page Changes**: The context menu disappears, and the text is copied to the clipboard (though this change is not visually observable on the page).
|
||
- **Possible Purpose**: The clear intention here is to copy the selected review text. This could be for documentation, analysis, or sharing purposes.
|
||
|
||
These actions are strictly based on the observations from the provided video segment, focusing solely on the interactions and their immediate effects on the webpage.
|
||
|
||
---
|
||
|
||
## Part 4
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I scroll down the webpage.
|
||
- **Page Changes**: The content shifts upward, revealing additional customer reviews below the initially visible ones.
|
||
- **Possible Purpose**: The likely intent is to view more customer feedback and gather a broader understanding of the product's performance and user satisfaction.
|
||
|
||
2. **Action**: I hover over a specific review.
|
||
- **Page Changes**: A tooltip or highlighted box appears, providing a closer view or emphasis on the text of the selected review.
|
||
- **Possible Purpose**: This action suggests an intent to focus on the details of this particular review, possibly because it contains important information or stands out for some reason (e.g., lower rating, detailed comment).
|
||
|
||
3. **Action**: I continue scrolling down the webpage.
|
||
- **Page Changes**: More reviews become visible as the previous ones move further up and out of immediate view.
|
||
- **Possible Purpose**: The continued scrolling indicates an intention to explore additional reviews, perhaps seeking a pattern in user experiences or looking for specific types of feedback (positive, negative, detailed usage scenarios).
|
||
|
||
Each action is performed methodically, focusing on gathering comprehensive information from the customer reviews section of the webpage. There are no clicks or selections of specific links or buttons observed in this segment; the interaction is primarily through scrolling and hovering.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "action_0",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search entire store here...\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_1",
|
||
"idx": 1,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search entire store here...\")Enter",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Image",
|
||
"idx": 2,
|
||
"action_repr": "frame.clicklocator(\"#product-item-info_38560\").get_by_role(\"link\", name=\"Image\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/catalogsearch/result/?q=EYZUTAK+phone+cases"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_12 Reviews",
|
||
"idx": 3,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"12 Reviews\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Page 2",
|
||
"idx": 4,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Page 2\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "text_N Randall",
|
||
"idx": 6,
|
||
"action_repr": "frame.clickget_by_text(\"N Randall\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Page 1",
|
||
"idx": 7,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Page 1\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/eyzutak-electroplated-magnetic-ring-holder-case-360-degree-with-rotation-metal-finger-ring-holder-magnet-car-holder-soft-silicone-shockproof-cover-for-iphone-7-iphone-8-iphone-se-2020-drak-green.html"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |