245 lines
14 KiB
Plaintext
245 lines
14 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
List out reviewers, if exist, who mention about good fingerprint resistant
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Action:** I click on the "12 Reviews" link.
|
||
- **Page Changes:** The page transitions to a section displaying customer reviews for the product.
|
||
- **Possible Purpose:** The likely intent is to read and evaluate the feedback from other customers to gain insights into the product's performance and reliability.
|
||
|
||
#### 2. **Action:** I scroll down through the customer reviews.
|
||
- **Page Changes:** The visible content updates to show more detailed reviews, including ratings, comments, and dates posted.
|
||
- **Possible Purpose:** The purpose is to thoroughly examine multiple reviews to understand common themes, such as product quality, ease of use, and customer satisfaction.
|
||
|
||
#### 3. **Action:** I hover over a specific review text.
|
||
- **Page Changes:** A tooltip or highlighted text appears, showing the exact content or metadata of the review being hovered over.
|
||
- **Possible Purpose:** This action aims to closely inspect the details of a particular review, possibly to verify its relevance or authenticity.
|
||
|
||
#### 4. **Action:** I click on a specific review titled "Review by Chris."
|
||
- **Page Changes:** The page may highlight or expand the selected review, bringing it into focus for easier reading.
|
||
- **Possible Purpose:** The intent is to read this specific review in detail, perhaps because it contains particularly relevant information or a detailed account of the product's usage.
|
||
|
||
---
|
||
|
||
### Summary of Observed Actions:
|
||
- **Initial Interaction:** I begin by accessing the customer reviews section via the "12 Reviews" link.
|
||
- **Exploration:** I then scroll through the reviews to get an overview of customer feedback.
|
||
- **Detailed Examination:** I hover over a specific review to inspect its details more closely.
|
||
- **Focused Reading:** Finally, I select a particular review ("Review by Chris") to read it in full detail.
|
||
|
||
Each step is performed with the clear objective of gathering comprehensive information about the product based on user experiences.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I hover over a specific review on the webpage.
|
||
- **Page Changes**: There are no immediate changes to the webpage; however, a tooltip or highlight appears over the text of the review, indicating interaction.
|
||
- **Possible Purpose**: The likely intent is to read or focus on the details of this particular review more closely. Hovering can help in emphasizing the text or preparing for further interaction, such as selecting or copying the text.
|
||
|
||
2. **Action**: I select a portion of the text within the highlighted review.
|
||
- **Page Changes**: The selected text becomes visually distinguished, typically by being highlighted in a different color (e.g., blue).
|
||
- **Possible Purpose**: The purpose of selecting the text could be to copy it for further use, such as pasting into another application, searching for specific information, or analyzing the content more closely.
|
||
|
||
3. **Action**: I right-click on the selected text.
|
||
- **Page Changes**: A context menu appears with various options such as "Copy," "Search with Google," "Inspect," etc.
|
||
- **Possible Purpose**: Right-clicking on the selected text is commonly done to access additional options related to the text. The most probable intent here is to choose an action from the context menu, such as copying the text.
|
||
|
||
4. **Action**: I click on the "Copy" option in the context menu.
|
||
- **Page Changes**: The context menu disappears, and there is no visible change on the webpage, but the text is now copied to the clipboard.
|
||
- **Possible Purpose**: The intent of copying the text is to transfer it to another location, such as a document, email, or search bar, for further processing or reference.
|
||
|
||
These actions are strictly based on the observations from the provided video segment and do not include any assumptions beyond what is directly observable.
|
||
|
||
---
|
||
|
||
## Part 3
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I scroll down the webpage.
|
||
- **Page Changes**: The content of the webpage shifts upward, revealing more customer reviews and ratings further down the page.
|
||
- **Possible Purpose**: The likely intent is to view additional customer feedback and gather more information about the product's performance and user satisfaction.
|
||
|
||
2. **Action**: I highlight a specific section of text within a review.
|
||
- **Page Changes**: The highlighted text becomes visually distinct from the rest of the page, drawing attention to the selected portion.
|
||
- **Possible Purpose**: The purpose appears to be to focus on particular details or comments within the review, possibly for closer examination or to extract specific information.
|
||
|
||
3. **Action**: I continue scrolling down the webpage.
|
||
- **Page Changes**: More reviews and ratings become visible as the previous content moves further up the page.
|
||
- **Possible Purpose**: The intent is likely to explore more user experiences and gather a broader perspective on the product's quality and functionality.
|
||
|
||
4. **Action**: I pause briefly over a specific review.
|
||
- **Page Changes**: There are no changes to the page itself, but the cursor remains stationary over the review.
|
||
- **Possible Purpose**: This action suggests an intent to read and comprehend the content of the review in detail, possibly to evaluate the reviewer's experience or note specific points mentioned.
|
||
|
||
These actions are strictly based on the observable elements within this video segment, focusing solely on the interactions and their immediate effects on the webpage.
|
||
|
||
---
|
||
|
||
## Part 4
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I hover over a specific review text on the webpage.
|
||
- **Page Changes**: There are no immediate changes to the webpage; however, the text under the cursor becomes highlighted, indicating it is selectable.
|
||
- **Possible Purpose**: The likely intent is to focus on or select this particular review text for further action, such as copying or examining it closely.
|
||
|
||
2. **Action**: I click and drag to select a portion of the review text.
|
||
- **Page Changes**: The selected text is now highlighted in a different color (typically blue), indicating it has been selected.
|
||
- **Possible Purpose**: The purpose of this action is to highlight the text, possibly in preparation for copying it or to draw attention to specific content within the review.
|
||
|
||
3. **Action**: I right-click on the selected text.
|
||
- **Page Changes**: A context menu appears with various options such as "Copy," "Search with Google," etc.
|
||
- **Possible Purpose**: The intent is to access additional options related to the selected text, such as copying the text or performing a search.
|
||
|
||
4. **Action**: I click on the "Copy" option from the context menu.
|
||
- **Page Changes**: The context menu disappears, and there is no visible change on the webpage, but the text is now copied to the clipboard.
|
||
- **Possible Purpose**: The purpose is to copy the selected text so it can be pasted elsewhere, such as into a document or another application.
|
||
|
||
These steps describe the precise actions taken within the provided video segment, focusing solely on the observable interactions with the webpage.
|
||
|
||
---
|
||
|
||
## Part 5
|
||
In this video segment, the following actions are observed:
|
||
|
||
1. **Action**: I highlight a section of text on the webpage.
|
||
- **Page Changes**: The highlighted text appears in a different color (typically yellow or another contrasting color), making it stand out from the rest of the text.
|
||
- **Possible Purpose**: The likely intent is to draw attention to specific content on the page, possibly for the purpose of copying, reviewing, or annotating the text.
|
||
|
||
2. **Action**: I right-click on the highlighted text.
|
||
- **Page Changes**: A context menu appears with various options such as "Copy," "Search with Google," "Inspect," etc.
|
||
- **Possible Purpose**: The action of right-clicking suggests an intention to perform an operation on the highlighted text, such as copying it or inspecting its properties.
|
||
|
||
3. **Action**: I select the "Copy" option from the context menu.
|
||
- **Page Changes**: The context menu disappears, and there is no immediate visual change on the page, but the text is now stored in the clipboard.
|
||
- **Possible Purpose**: The purpose of copying the text is likely to transfer it to another location, such as a document, email, or another part of the webpage.
|
||
|
||
4. **Action**: I move the cursor to a different area of the webpage.
|
||
- **Page Changes**: The cursor changes position, but there are no other visible changes to the page.
|
||
- **Possible Purpose**: This action might be preparatory, indicating an intention to paste the copied text or interact with another element on the page.
|
||
|
||
5. **Action**: I click on a text box or input field.
|
||
- **Page Changes**: The text box or input field becomes active, indicated by a blinking cursor inside it.
|
||
- **Possible Purpose**: Activating the text box suggests an intention to enter or paste text into this field.
|
||
|
||
6. **Action**: I paste the copied text into the active text box.
|
||
- **Page Changes**: The previously copied text now appears within the text box.
|
||
- **Possible Purpose**: The purpose of pasting the text is to insert the copied content into the text box, possibly for submission, editing, or further processing.
|
||
|
||
These steps describe the precise actions taken within the provided video segment, focusing solely on the observable interactions and their immediate effects on the webpage.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "link_12 Reviews",
|
||
"idx": 1,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"12 Reviews\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/3-pack-samsung-galaxy-s6-screen-protector-nearpow-tempered-glass-screen-protector-with-9h-hardness-crystal-clear-easy-bubble-free-installation-scratch-resist.html"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/3-pack-samsung-galaxy-s6-screen-protector-nearpow-tempered-glass-screen-protector-with-9h-hardness-crystal-clear-easy-bubble-free-installation-scratch-resist.html"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Page 2",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Page 2\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/3-pack-samsung-galaxy-s6-screen-protector-nearpow-tempered-glass-screen-protector-with-9h-hardness-crystal-clear-easy-bubble-free-installation-scratch-resist.html#reviews"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7770/3-pack-samsung-galaxy-s6-screen-protector-nearpow-tempered-glass-screen-protector-with-9h-hardness-crystal-clear-easy-bubble-free-installation-scratch-resist.html#reviews"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |