252 lines
13 KiB
Plaintext
252 lines
13 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
Show the least expensive switch card holder with a minimum storage capacity of 15 cards.
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Part 1: Initial Actions on the Homepage
|
||
|
||
#### Action:
|
||
I click on the search bar located at the top center of the webpage.
|
||
|
||
- **Page Changes:**
|
||
The search bar becomes active, and a placeholder text reading "Search entire store here..." is highlighted. A blinking cursor appears inside the search bar, indicating it is ready for input.
|
||
|
||
- **Possible Purpose:**
|
||
The likely intent is to initiate a product search by entering specific keywords into the search bar. This action prepares the interface for typing the desired search terms.
|
||
|
||
#### Action:
|
||
I type "switch card holder" into the active search bar.
|
||
|
||
- **Page Changes:**
|
||
As I type, the text "switch card holder" replaces the placeholder text in the search bar. No other immediate changes occur on the page until the search is executed.
|
||
|
||
- **Possible Purpose:**
|
||
The purpose is to find products related to "switch card holder." Typing the query is a necessary step before executing the search to retrieve relevant results.
|
||
|
||
#### Action:
|
||
I press the Enter key or click the search icon next to the search bar.
|
||
|
||
- **Page Changes:**
|
||
The webpage transitions from the homepage to a new page displaying search results for "switch card holder." The new page lists various products that match the search query, each with images, names, prices, and an "Add to Cart" button.
|
||
|
||
- **Possible Purpose:**
|
||
The intent is to execute the search and view the available products that match the entered keywords. This action allows me to browse and potentially select a product to purchase.
|
||
|
||
### Summary of Part 1:
|
||
In this segment, I initiated a product search on the website by activating the search bar, entering the query "switch card holder," and executing the search. The result was a transition to a new page showcasing relevant products. Each step was methodical, aimed at finding and viewing products matching the specified search terms.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the "Add to Cart" button for the product titled "Game Card Holder Storage Case for Nintendo Switch Games or PS Vita Game Case or SD Memory Cards, Black."
|
||
- **Page Changes:** A notification appears confirming that the item has been added to the cart.
|
||
- **Possible Purpose:** The likely intent is to add this specific product to the shopping cart for potential purchase.
|
||
|
||
2. **Action:** I hover over the product image of the "AxiGear 34-Slot SD Card Holder Hard Case Holds SD/SDHC/SDXC Memory Cards (NOT miniSD or microSD)."
|
||
- **Page Changes:** There are no immediate changes to the page; however, the cursor changes to a hand icon, indicating interactivity.
|
||
- **Possible Purpose:** The purpose is likely to view more details about the product, such as a larger image or additional information, which might be revealed upon further interaction (e.g., clicking).
|
||
|
||
3. **Action:** I click on the product image of the "AxiGear 34-Slot SD Card Holder Hard Case Holds SD/SDHC/SDXC Memory Cards (NOT miniSD or microSD)."
|
||
- **Page Changes:** The webpage transitions to the product detail page for the selected item, displaying detailed information, images, and options related to the product.
|
||
- **Possible Purpose:** The intent is to gather more detailed information about the product, including specifications, reviews, and purchasing options, before making a decision to buy.
|
||
|
||
### Summary:
|
||
- The actions involve adding a specific product to the cart and then exploring another product in detail.
|
||
- The first action successfully adds an item to the cart, as confirmed by the notification.
|
||
- The second and third actions focus on gathering more information about a different product, leading to the product detail page.
|
||
- These steps suggest a shopping process where items are being evaluated for potential purchase.
|
||
|
||
---
|
||
|
||
## Part 3
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Action:** I click on the "Add to Cart" button for the AxiGear 34 Slot SD Card Holder product.
|
||
- **Page Changes:** After clicking, a notification or confirmation message appears, indicating that the item has been successfully added to the cart. The page remains on the product details view.
|
||
- **Possible Purpose:** The likely intent is to add the selected product to the shopping cart for potential purchase.
|
||
|
||
#### 2. **Action:** I navigate back to the previous page using the browser's back button or a navigation link.
|
||
- **Page Changes:** The webpage transitions from the product details page to a category or search results page displaying various game card holder storage cases.
|
||
- **Possible Purpose:** The purpose is to return to a broader selection of products, possibly to continue shopping or compare other items.
|
||
|
||
#### 3. **Action:** I scroll down the page to view more products.
|
||
- **Page Changes:** As I scroll, additional product listings become visible, each with images, names, prices, and "Add to Cart" buttons.
|
||
- **Possible Purpose:** The intent is to browse through more available options and potentially find another item of interest.
|
||
|
||
#### 4. **Action:** I click on a specific product titled "Game Card Holder Storage Case for Nintendo Switch Games or PS Vita Game Case or SD Memory Cards, Black."
|
||
- **Page Changes:** The webpage transitions to the detailed view of the selected product. This new page includes a larger image of the product, a description, price, and an "Add to Cart" button.
|
||
- **Possible Purpose:** The goal is to view the details of this particular product, such as its features, specifications, and price, before making a decision to purchase.
|
||
|
||
#### 5. **Action:** I scroll down on the product details page.
|
||
- **Page Changes:** Scrolling reveals more information about the product, including a detailed description, related search terms, and customer reviews.
|
||
- **Possible Purpose:** The intent is to gather more information about the product to make an informed purchasing decision.
|
||
|
||
#### 6. **Action:** I click on the "Add to Cart" button for the "Game Card Holder Storage Case for Nintendo Switch Games or PS Vita Game Case or SD Memory Cards, Black."
|
||
- **Page Changes:** A confirmation message appears, indicating that the item has been added to the cart. The page remains on the product details view.
|
||
- **Possible Purpose:** The purpose is to add this selected product to the shopping cart, moving forward with the intention to purchase it.
|
||
|
||
### Summary
|
||
In this video segment, I perform a series of actions focused on adding products to the shopping cart. Initially, I add an SD card holder to the cart, then navigate back to browse more options. I select a game card holder storage case, review its details, and add it to the cart as well. Each action is aimed at building a collection of desired items in the cart for potential checkout.
|
||
|
||
---
|
||
|
||
## Part 4
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I hover over the text "Nintendo Switch Game Case" within a highlighted section of the webpage.
|
||
- **Page Changes**: There are no immediate changes to the webpage as a result of this action; the page remains static.
|
||
- **Possible Purpose**: The likely intent is to either read the text more closely or prepare to interact with it, such as clicking or selecting it for further information.
|
||
|
||
2. **Action**: I click on the text "Nintendo Switch Game Case."
|
||
- **Page Changes**: The webpage does not visibly change in this segment, suggesting that the click might be part of a selection process or an interaction that doesn't immediately alter the page view.
|
||
- **Possible Purpose**: The purpose could be to select this specific item from a list or menu, potentially to view more details about the product or to add it to a cart if this is an e-commerce site.
|
||
|
||
These actions are strictly based on the observations from the provided video segment, focusing solely on the interactions and their immediate observable effects without additional context or assumptions.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "action_0",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search entire store here...\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_1",
|
||
"idx": 1,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search entire store here...\")Enter",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Show",
|
||
"idx": 2,
|
||
"action_repr": "frame.selectOptionget_by_label(\"Show\")",
|
||
"before": {
|
||
"url": "about:blank"
|
||
},
|
||
"after": {
|
||
"url": "about:blank"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_3",
|
||
"idx": 3,
|
||
"action_repr": "frame.selectOptionlocator(\"#maincontent div\").filter(has_text=\"Search results for: 'switch card holder'\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/catalogsearch/result/index/?q=switch%20card%20holder&product_list_limit=36"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/catalogsearch/result/index/?q=switch%20card%20holder&product_list_limit=36"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Image",
|
||
"idx": 5,
|
||
"action_repr": "frame.clicklocator(\"#product-item-info_18448\").get_by_role(\"link\", name=\"Image\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/axigear-34-slot-sd-card-holder-hard-case-holds-sd-sdhc-sdxc-sduc-memory-cards-not-minisd-or-microsd.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/catalogsearch/result/index/?q=switch%20card%20holder&product_list_limit=36"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |