543 lines
25 KiB
Plaintext
543 lines
25 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
Display the list of issues in the kkroening/ffmpeg-python repository that have labels related to questions
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Initial State**
|
||
- **Action:** The video begins with a view of a GitLab projects page. The page displays a list of projects under "Yours," with details such as project names, descriptions, and update times.
|
||
- **Page Changes:** No immediate changes occur as this is the starting point.
|
||
- **Possible Purpose:** The initial state sets the context for navigating through the user's projects on GitLab.
|
||
|
||
#### 2. **Hovering Over Projects**
|
||
- **Action:** I move the cursor over several project entries in the list.
|
||
- **Page Changes:** As the cursor hovers over each project, a tooltip appears, displaying the full URL of the project.
|
||
- **Possible Purpose:** Hovering over the projects likely aims to review the project URLs or gather more information about each project before selecting one.
|
||
|
||
#### 3. **Scrolling Down the Page**
|
||
- **Action:** I scroll down the page using the mouse wheel.
|
||
- **Page Changes:** The list of projects moves upward, revealing additional projects further down the list.
|
||
- **Possible Purpose:** Scrolling down is intended to view more projects that are not initially visible on the screen.
|
||
|
||
#### 4. **Hovering Over Additional Projects**
|
||
- **Action:** After scrolling, I continue to hover over different project entries.
|
||
- **Page Changes:** Similar to the previous hovering action, tooltips appear showing the full URLs of the newly visible projects.
|
||
- **Possible Purpose:** This continued hovering suggests an ongoing review of project details, possibly to locate a specific project or assess available options.
|
||
|
||
#### 5. **Stopping at a Specific Project**
|
||
- **Action:** I stop scrolling and hover over a project named "Byte Blazea11yproject.contributor.me / cloud-to-butt."
|
||
- **Page Changes:** The tooltip for this project appears, displaying its full URL.
|
||
- **Possible Purpose:** Pausing at this specific project indicates interest in it, perhaps for further interaction or selection.
|
||
|
||
#### 6. **Clicking on the Project**
|
||
- **Action:** I click on the project "Byte Blazea11yproject.contributor.me / cloud-to-butt."
|
||
- **Page Changes:** The page transitions from the projects list to the detailed view of the selected project. The new page shows the project's repository files, commit history, and other related information.
|
||
- **Possible Purpose:** Clicking on the project is to access its detailed page, likely for viewing its contents, making edits, or performing other project-specific actions.
|
||
|
||
### Summary
|
||
In this video segment, I begin by reviewing a list of projects on my GitLab page. I hover over various projects to see their URLs, scroll down to view more projects, and then select a specific project ("Byte Blazea11yproject.contributor.me / cloud-to-butt") by clicking on it. This sequence of actions suggests a focused effort to locate and access a particular project for further interaction.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the "Personal" tab located under the "Projects" section.
|
||
- **Page Changes:** The page updates to display a message: "This user doesn't have any personal projects."
|
||
- **Possible Purpose:** The likely intent is to view the list of personal projects associated with the current user account.
|
||
|
||
2. **Action:** I click on the search bar at the top of the page, which contains the placeholder text "Search GitLab."
|
||
- **Page Changes:** The search bar becomes active, allowing for text input.
|
||
- **Possible Purpose:** The intention is to initiate a search within GitLab, possibly to find specific repositories, users, or other resources.
|
||
|
||
3. **Action:** I type "kkroening" into the search bar.
|
||
- **Page Changes:** As I type, suggestions appear below the search bar, including options like "Users," "Repositories," and specific names matching the query.
|
||
- **Possible Purpose:** The goal is to search for a user or repository related to "kkroening."
|
||
|
||
4. **Action:** I select the "kkroening" suggestion from the dropdown list.
|
||
- **Page Changes:** The search results update to show content related to "kkroening."
|
||
- **Possible Purpose:** The aim is to navigate to the profile or projects associated with the user "kkroening."
|
||
|
||
### Summary:
|
||
In this video segment, I navigated from viewing personal projects (which resulted in a message indicating no personal projects) to using the search functionality. I activated the search bar, entered "kkroening," and selected the corresponding suggestion to find relevant content. Each step was methodical, focusing on utilizing the search feature to locate specific information within GitLab.
|
||
|
||
---
|
||
|
||
## Part 3
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Initial State**
|
||
- **Action:** The video starts with a search page where the term "kkroening" has been entered into the search bar.
|
||
- **Page Changes:** The search results display counts for various categories: Projects (0), Issues (12), Merge requests (15), Milestones (0), and Users (1). A message states, "We couldn't find any projects matching kkroening."
|
||
- **Possible Purpose:** The initial action is to locate content related to "kkroening" within the platform.
|
||
|
||
#### 2. **Navigating to the User Profile**
|
||
- **Action:** I click on the "Users" category, which shows a count of 1.
|
||
- **Page Changes:** The page transitions to display the user profile for "Karl Kroening," including details such as username (@kkroening), user ID, membership date, and activity overview.
|
||
- **Possible Purpose:** The intent is to access the specific user profile associated with "kkroening" to view more detailed information about this user.
|
||
|
||
#### 3. **Exploring the User's Personal Projects**
|
||
- **Action:** I click on the "Personal projects" tab within the user profile.
|
||
- **Page Changes:** The page updates to show a list of personal projects associated with the user. One project, "ffmpeg-python," is highlighted.
|
||
- **Possible Purpose:** The goal is to examine the personal projects of the user, specifically focusing on the "ffmpeg-python" project.
|
||
|
||
#### 4. **Accessing the Project Page**
|
||
- **Action:** I click on the "ffmpeg-python" project link.
|
||
- **Page Changes:** The page navigates to the "ffmpeg-python" project page, displaying project details such as description, commits, branches, tags, and project storage information.
|
||
- **Possible Purpose:** The intention is to delve into the specifics of the "ffmpeg-python" project, possibly to gather information or interact with the project resources.
|
||
|
||
#### 5. **Viewing Project Issues**
|
||
- **Action:** I click on the "Issues" tab in the left sidebar menu of the project page.
|
||
- **Page Changes:** The page transitions to the issues section of the "ffmpeg-python" project, showing a list of issues. A message indicates, "There are no open issues."
|
||
- **Possible Purpose:** The aim is to check for any open issues related to the "ffmpeg-python" project, perhaps to assess the project's current status or to find areas that might need attention.
|
||
|
||
### Summary
|
||
In this video segment, I perform a series of actions to locate and explore content related to "kkroening" within a platform. Specifically, I:
|
||
1. Search for "kkroening" and navigate to the user profile.
|
||
2. Examine the user's personal projects and select the "ffmpeg-python" project.
|
||
3. Review the details of the "ffmpeg-python" project and check for any open issues.
|
||
|
||
Each step is executed with the apparent purpose of gathering detailed information about the user and their projects, particularly focusing on the "ffmpeg-python" project and its issue status.
|
||
|
||
---
|
||
|
||
## Part 4
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the search bar located at the top of the issues page.
|
||
- **Page Changes:** The search bar becomes active, and a dropdown menu appears below it, displaying various filter options such as "Label," "Milestone," "Author," etc.
|
||
- **Possible Purpose:** The likely intent is to filter or search for specific issues based on certain criteria.
|
||
|
||
2. **Action:** I type `label:question` into the search bar.
|
||
- **Page Changes:** As I type, the dropdown menu updates to show suggestions related to the text I've entered. The phrase `label:question` is highlighted in the suggestions.
|
||
- **Possible Purpose:** The purpose is to filter the list of issues to only show those that have the "question" label.
|
||
|
||
3. **Action:** I select the `label:question` suggestion from the dropdown menu.
|
||
- **Page Changes:** The search query `label:question` is applied, and the list of issues updates to display only those tagged with the "question" label.
|
||
- **Possible Purpose:** This action narrows down the visible issues to those specifically marked as questions, making it easier to focus on this subset.
|
||
|
||
4. **Action:** I click on the "Label" filter option in the dropdown menu.
|
||
- **Page Changes:** A new interface appears, showing a list of available labels with checkboxes next to them.
|
||
- **Possible Purpose:** The intent is to further refine the issue filter by selecting or deselecting specific labels.
|
||
|
||
5. **Action:** I scroll through the list of labels and click on the checkbox next to the "question" label.
|
||
- **Page Changes:** The "question" label is selected, and the issues list updates to reflect this selection if it wasn't already filtered.
|
||
- **Possible Purpose:** This ensures that the filter for the "question" label is explicitly set, confirming the focus on issues tagged as questions.
|
||
|
||
6. **Action:** I close the label selection interface by clicking outside of it or on a close button.
|
||
- **Page Changes:** The label selection interface closes, and the main issues list remains filtered by the "question" label.
|
||
- **Possible Purpose:** To return to the main view of the filtered issues list for further review or action.
|
||
|
||
### Summary:
|
||
In this video segment, I interact with the search and filter functionalities on the GitLab issues page to focus on issues labeled as "question." My actions include typing a specific filter query, selecting suggested filters, and using the label selection interface to ensure the correct filter is applied. The page responds by updating the list of issues to match the specified criteria, allowing for targeted viewing of relevant issues.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "action_0",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_1",
|
||
"idx": 1,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_2",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_3",
|
||
"idx": 3,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_4",
|
||
"idx": 4,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_5",
|
||
"idx": 5,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Filter by name\")ArrowRight",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeg-python&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_6",
|
||
"idx": 6,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Filter by name\")ArrowRight",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_7",
|
||
"idx": 7,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Filter by name\")ArrowRight",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_8",
|
||
"idx": 8,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Filter by name\")ArrowRight",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_9",
|
||
"idx": 9,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Filter by name\")ArrowRight",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?sort=name_asc&name=kkroening%2Fffmpeython&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_10",
|
||
"idx": 10,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Filter by name\")Enter",
|
||
"before": {
|
||
"url": "about:blank"
|
||
},
|
||
"after": {
|
||
"url": "about:blank"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_11",
|
||
"idx": 11,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "about:blank"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_12",
|
||
"idx": 12,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "text_Yours 23 Starred 3 Explore Topics",
|
||
"idx": 13,
|
||
"action_repr": "frame.clickget_by_text(\"Yours 23 Starred 3 Explore Topics\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_14",
|
||
"idx": 14,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_15",
|
||
"idx": 15,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Filter by name\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_16",
|
||
"idx": 16,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Filter by name\")Control+c",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_17",
|
||
"idx": 17,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search GitLab\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_18",
|
||
"idx": 18,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search GitLab\")Enter",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/?name=kkroening&sort=name_asc"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Users 1",
|
||
"idx": 19,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Users 1\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=kkroening&nav_source=navbar"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=kkroening&nav_source=navbar"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Karl Kroening @kkroening",
|
||
"idx": 20,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Karl Kroening @kkroening\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?scope=users&search=kkroening"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?scope=users&search=kkroening"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Personal projects",
|
||
"idx": 21,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Personal projects\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_ffmpeg-python",
|
||
"idx": 22,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"ffmpeg-python\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/kkroening/projects"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_23",
|
||
"idx": 23,
|
||
"action_repr": "frame.clicklocator(\"a\").filter(has_text=\"Issues 402\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "filtered-search-token-segment",
|
||
"idx": 24,
|
||
"action_repr": "frame.clickget_by_test_id(\"filtered-search-token-segment\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "menuitem_Label",
|
||
"idx": 25,
|
||
"action_repr": "frame.clickget_by_role(\"menuitem\", name=\"Label\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "menuitem_= is",
|
||
"idx": 26,
|
||
"action_repr": "frame.clickget_by_role(\"menuitem\", name=\"= is\", exact=True)",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "menuitem_question",
|
||
"idx": 27,
|
||
"action_repr": "frame.clickget_by_role(\"menuitem\", name=\"question\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "search-button",
|
||
"idx": 28,
|
||
"action_repr": "frame.clickget_by_test_id(\"search-button\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/kkroening/ffmpeg-python/-/issues"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |