233 lines
12 KiB
Plaintext
233 lines
12 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
List all opened issues requesting new features
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Initial State**
|
||
- **Action:** The video starts with a view of a GitLab project page for the "metaseq" repository.
|
||
- **Page Changes:** The page displays various sections such as Project information, Repository, Issues, Merge requests, etc., on the left sidebar. The main content area shows an overview of the repository, including commits, branches, and project storage details.
|
||
- **Possible Purpose:** The initial state sets the context for navigating within the GitLab project.
|
||
|
||
#### 2. **Navigating to the Issues Section**
|
||
- **Action:** I click on the "Issues" link in the left sidebar.
|
||
- **Page Changes:** The page transitions to the Issues section of the "metaseq" project. This section lists open and closed issues, with options to filter by status (Open, Closed, All).
|
||
- **Possible Purpose:** The intent is to access and manage the issues related to the project, possibly to review, create, or update them.
|
||
|
||
#### 3. **Observing the Issues List**
|
||
- **Action:** I observe the list of issues displayed under the "Open" tab.
|
||
- **Page Changes:** The issues are listed with details such as title, assignee, labels, and last update time. There are 95 open issues visible.
|
||
- **Possible Purpose:** The purpose is to get an overview of the current open issues to identify any specific issue that needs attention.
|
||
|
||
#### 4. **Using the Search Function**
|
||
- **Action:** I click on the search bar located above the list of issues.
|
||
- **Page Changes:** The search bar becomes active, allowing text input to filter the issues.
|
||
- **Possible Purpose:** The intent is to narrow down the list of issues using specific keywords or criteria for more targeted viewing.
|
||
|
||
#### 5. **Typing in the Search Bar**
|
||
- **Action:** I type a query into the search bar.
|
||
- **Page Changes:** As I type, the list of issues dynamically updates to show only those matching the search criteria.
|
||
- **Possible Purpose:** The purpose is to find a specific issue or set of issues that match the entered keywords, making it easier to locate relevant information quickly.
|
||
|
||
#### 6. **Reviewing Filtered Results**
|
||
- **Action:** I review the filtered list of issues resulting from the search query.
|
||
- **Page Changes:** The issues list now contains only the entries that match the search terms, providing a focused view.
|
||
- **Possible Purpose:** The intent is to analyze the filtered results to find the exact issue or gather information about a particular topic.
|
||
|
||
#### 7. **Hovering Over Issue Details**
|
||
- **Action:** I hover over one of the issues in the filtered list.
|
||
- **Page Changes:** Hovering over an issue may reveal additional details or options related to that specific issue, though no significant changes are observed in the video.
|
||
- **Possible Purpose:** The purpose is to get a quick preview or additional context about the issue without clicking into it, aiding in decision-making or prioritization.
|
||
|
||
### Summary
|
||
In this video segment, I navigate from the main project page to the Issues section of a GitLab repository. I use the search functionality to filter the list of open issues, focusing on specific criteria. The actions are aimed at efficiently locating and reviewing relevant issues within the project. Each step is performed methodically to achieve a clear and targeted view of the project's issue management.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the "Label =" dropdown menu.
|
||
- **Page Changes:** A dropdown list appears, displaying various labels such as `enhancement`, `bug`, `question`, etc.
|
||
- **Possible Purpose:** The likely intent is to filter or assign a specific label to the issues displayed on the page.
|
||
|
||
2. **Action:** I type `-enhancement` into the search box within the dropdown menu.
|
||
- **Page Changes:** The dropdown list updates to show options that match the input. Specifically, it highlights the `enhancement` label.
|
||
- **Possible Purpose:** The purpose is to either exclude issues labeled as `enhancement` from the current view or to prepare for removing this label from selected issues.
|
||
|
||
3. **Action:** I select the `enhancement` label from the dropdown list.
|
||
- **Page Changes:** The label `enhancement` is added to the filter criteria, and the issues list updates to reflect this change. Issues that do not meet the new filter criteria may be hidden or altered in display.
|
||
- **Possible Purpose:** The intent is to apply the `enhancement` label filter, either to view only enhancement-related issues or to manage these issues specifically.
|
||
|
||
4. **Action:** I click on the "X" button next to the `enhancement` label in the filter bar.
|
||
- **Page Changes:** The `enhancement` label is removed from the filter criteria, and the issues list reverts to its previous state, showing all issues regardless of the `enhancement` label.
|
||
- **Possible Purpose:** The purpose is to remove the `enhancement` label filter, thereby restoring the original view of all issues without this specific label constraint.
|
||
|
||
5. **Action:** I begin typing `get_by_test_id("search-dataset", name="telemetry")` into the search box within the dropdown menu.
|
||
- **Page Changes:** As I type, the dropdown list dynamically updates to suggest matching labels or entries based on the text input. However, since this is not a standard label name, no exact matches appear, but related suggestions might be shown.
|
||
- **Possible Purpose:** The intent might be to search for a specific label or issue related to "telemetry" or "search-dataset," although the exact phrase does not correspond to any visible label options.
|
||
|
||
6. **Action:** I stop typing and do not select any further options from the dropdown.
|
||
- **Page Changes:** The dropdown remains open with the typed text visible, but no additional changes occur to the issues list.
|
||
- **Possible Purpose:** The purpose could be to pause the search or filtering process, possibly to rethink the search query or to manually review the issues list without applying a new filter.
|
||
|
||
### Summary:
|
||
In this video segment, I interact with the label filtering system on the GitLab issues page. I attempt to filter issues using the `enhancement` label, then remove this filter, and finally try an unconventional search query. These actions suggest a focus on managing and reviewing specific categories of issues, though the final search query does not yield standard results.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "action_0",
|
||
"idx": 0,
|
||
"action_repr": "frame.clicklocator(\"a\").filter(has_text=\"Issues 95\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "filtered-search-term-input",
|
||
"idx": 1,
|
||
"action_repr": "frame.clickget_by_test_id(\"filtered-search-term-input\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "menuitem_Label",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_role(\"menuitem\", name=\"Label\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "menuitem_= is",
|
||
"idx": 3,
|
||
"action_repr": "frame.clickget_by_role(\"menuitem\", name=\"= is\", exact=True)",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "menuitem_enhancement",
|
||
"idx": 4,
|
||
"action_repr": "frame.clickget_by_role(\"menuitem\", name=\"enhancement\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "search-button",
|
||
"idx": 5,
|
||
"action_repr": "frame.clickget_by_test_id(\"search-button\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/root/metaseq/-/issues"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |