621 lines
30 KiB
Plaintext
621 lines
30 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
Check out my todos
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Action:** I click on the "Yours" tab.
|
||
- **Page Changes:** The page updates to display a list of projects under the "Yours" category, showing 23 projects.
|
||
- **Possible Purpose:** To view and manage the projects that I own or contribute to.
|
||
|
||
#### 2. **Action:** I hover over the "Starred" tab.
|
||
- **Page Changes:** No immediate change occurs; the "Starred" tab is highlighted, indicating it is the next target for interaction.
|
||
- **Possible Purpose:** To prepare for selecting the "Starred" tab, likely to view projects I have marked as important.
|
||
|
||
#### 3. **Action:** I click on the "Starred" tab.
|
||
- **Page Changes:** The page updates to display a list of projects under the "Starred" category, showing 3 projects.
|
||
- **Possible Purpose:** To view and access the projects I have specifically starred for easy reference.
|
||
|
||
#### 4. **Action:** I move the cursor to the top-right corner of the screen and click on the user profile icon (avatar).
|
||
- **Page Changes:** A dropdown menu appears with options such as "Edit status," "Edit profile," "Preferences," and "Sign out."
|
||
- **Possible Purpose:** To access account-related settings or preferences from the user profile menu.
|
||
|
||
#### 5. **Action:** I click on the "Preferences" option in the dropdown menu.
|
||
- **Page Changes:** The page transitions to the Preferences settings page, where various user preference options are displayed.
|
||
- **Possible Purpose:** To modify or review personal settings and preferences related to the account.
|
||
|
||
### Summary
|
||
In this video segment, I navigate through different sections of the GitLab interface:
|
||
1. Switch from the "Yours" tab to the "Starred" tab to view specific projects.
|
||
2. Access the user profile menu by clicking the avatar icon.
|
||
3. Select the "Preferences" option to manage account settings. Each action is deliberate, focusing on navigating and customizing the user experience within GitLab.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Initial State**
|
||
- **Action:** The video begins with a view of a GitLab dashboard, specifically the "Projects" section under "Yours." The projects list is visible, showing various repositories with details like names, descriptions, and update times.
|
||
- **Page Changes:** No immediate changes occur as this is the starting point.
|
||
- **Possible Purpose:** The initial state sets the context for managing or reviewing personal projects on GitLab.
|
||
|
||
#### 2. **Hovering Over the User Profile Icon**
|
||
- **Action:** I move the cursor to the top-right corner of the screen and hover over the user profile icon (a circular icon with an avatar).
|
||
- **Page Changes:** A dropdown menu appears, displaying options such as "Edit status," "Edit profile," "Preferences," and "Sign out."
|
||
- **Possible Purpose:** Hovering over the profile icon is likely to access account-related settings or to sign out.
|
||
|
||
#### 3. **Clicking the User Profile Icon**
|
||
- **Action:** I click on the user profile icon.
|
||
- **Page Changes:** The page transitions to the user's profile overview. This new page shows sections like "Activity," "Personal projects," and other user-specific details.
|
||
- **Possible Purpose:** Clicking the profile icon navigates to the user's profile page, allowing for a detailed view of recent activities and managed projects.
|
||
|
||
#### 4. **Scrolling Down on the Profile Page**
|
||
- **Action:** I scroll down the profile page.
|
||
- **Page Changes:** As I scroll, more content becomes visible, including a list of "Personal projects" with project names, descriptions, and update statuses.
|
||
- **Possible Purpose:** Scrolling down is to review additional information or projects listed on the profile page.
|
||
|
||
#### 5. **Hovering Over a Project in the "Personal Projects" Section**
|
||
- **Action:** I move the cursor over a specific project in the "Personal projects" list.
|
||
- **Page Changes:** A tooltip or additional details about the project might appear, though this is not explicitly shown in the segment.
|
||
- **Possible Purpose:** Hovering over a project could be to get a quick preview or additional information about that specific project.
|
||
|
||
#### 6. **End of Segment**
|
||
- **Action:** The video segment ends with the cursor still over the "Personal projects" section.
|
||
- **Page Changes:** No further changes occur as the segment concludes.
|
||
- **Possible Purpose:** The final state suggests a focus on reviewing personal projects, possibly to select one for further action.
|
||
|
||
### Summary
|
||
In this video segment, I navigate from the "Projects" dashboard to my user profile page by interacting with the profile icon. I then scroll through the profile page to review my personal projects, indicating a focus on managing or examining these repositories. Each action is deliberate, moving from a general projects view to a more personalized overview.
|
||
|
||
---
|
||
|
||
## Part 3
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Action: I click on the search bar at the top of the GitLab page.**
|
||
- **Page Changes:** The search bar becomes active, and a dropdown menu appears below it, displaying recent searches or suggestions.
|
||
- **Possible Purpose:** To initiate a search within GitLab, likely to find specific projects, users, or issues.
|
||
|
||
#### 2. **Action: I type "todo" into the search bar.**
|
||
- **Page Changes:** As I type, the dropdown menu updates dynamically to show search suggestions related to the term "todo." These suggestions include projects, users, and other relevant items containing the word "todo."
|
||
- **Possible Purpose:** To locate specific items (e.g., projects, users) that are associated with the keyword "todo."
|
||
|
||
#### 3. **Action: I use the arrow keys on my keyboard to navigate through the search suggestions.**
|
||
- **Page Changes:** The highlighted suggestion in the dropdown menu changes as I press the up or down arrow keys, indicating different options.
|
||
- **Possible Purpose:** To select a specific suggestion from the list without using the mouse, enhancing navigation efficiency.
|
||
|
||
#### 4. **Action: I press the Enter key while a suggestion is highlighted.**
|
||
- **Page Changes:** The webpage transitions to display the details of the selected suggestion. If a project was selected, the project's main page is shown; if a user was selected, the user's profile page is displayed.
|
||
- **Possible Purpose:** To confirm the selection of a specific search result and view its detailed information.
|
||
|
||
---
|
||
|
||
### Summary of Observed Actions:
|
||
- I initiated a search by clicking on the search bar and typing "todo."
|
||
- I navigated through the search suggestions using the arrow keys.
|
||
- I confirmed my selection by pressing Enter, leading to the display of the chosen item's details.
|
||
|
||
Each action is performed with the intent to efficiently locate and access specific content related to the keyword "todo" within the GitLab platform.
|
||
|
||
---
|
||
|
||
## Part 4
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Action**: I click on the search bar at the top of the GitLab page.
|
||
- **Page Changes**: The search bar becomes active, and a dropdown menu appears below it, displaying recent searches and suggestions.
|
||
- **Possible Purpose**: The likely intent is to initiate a search within GitLab, possibly to find specific projects, issues, or users.
|
||
|
||
#### 2. **Action**: I type "my todo" into the search bar.
|
||
- **Page Changes**: As I type, the dropdown menu updates with search suggestions related to the text "my todo." The suggestions include projects and users that match the entered text.
|
||
- **Possible Purpose**: The purpose is to narrow down the search results to items specifically related to "my todo," which could be a project name, a user, or a keyword within GitLab.
|
||
|
||
#### 3. **Action**: I press the Enter key while the search bar is active.
|
||
- **Page Changes**: The page transitions to the search results page. The results display a list of projects and users that match the query "my todo." The projects are listed with their names, descriptions, and additional details like the number of stars and forks.
|
||
- **Possible Purpose**: The intent is to view all the search results for "my todo" to find the specific item I am looking for.
|
||
|
||
#### 4. **Action**: I hover over the first project in the search results.
|
||
- **Page Changes**: A tooltip appears, showing a selector path for the project, such as `get_by_text("Byte Blazea11project.contributor.me / TODO")`.
|
||
- **Possible Purpose**: The purpose might be to inspect the project more closely or to use the selector path for automation or testing purposes.
|
||
|
||
#### 5. **Action**: I click on the first project in the search results.
|
||
- **Page Changes**: The page navigates to the project's main page. The project's details, including the README, files, and other sections like Issues, Merge Requests, and Wiki, are displayed.
|
||
- **Possible Purpose**: The intent is to access the project's main page to view its contents, possibly to find specific information or to work on the project.
|
||
|
||
#### 6. **Action**: I scroll down on the project's main page.
|
||
- **Page Changes**: The content of the project page scrolls, revealing more sections such as "Authors and acknowledgment," "Test and Deploy," and "Project status."
|
||
- **Possible Purpose**: The purpose is to explore different sections of the project to gather more information or to find a specific section of interest.
|
||
|
||
#### 7. **Action**: I hover over the "Settings" link in the left sidebar menu.
|
||
- **Page Changes**: No immediate change occurs, but the "Settings" link is highlighted, indicating it is being hovered over.
|
||
- **Possible Purpose**: The intent might be to access the project settings to configure or manage the project's options.
|
||
|
||
### Summary
|
||
In this video segment, I perform a series of actions to search for and navigate to a specific project within GitLab. I start by using the search bar to find projects related to "my todo," then select and explore the first project in the search results. Finally, I scroll through the project's main page and hover over the "Settings" link, likely to gather more information or manage the project's settings. Each action is focused on locating and accessing specific information within GitLab.
|
||
|
||
---
|
||
|
||
## Part 5
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the "Monitor" menu item in the left sidebar.
|
||
- **Page Changes:** The page transitions to the "Monitor" section of the GitLab project, which includes tabs for "Metrics," "System Info," "Error Tracking," and "Alerts."
|
||
- **Possible Purpose:** The likely intent is to access monitoring tools and data related to the project, such as performance metrics or error logs, to assess the project's status or troubleshoot issues.
|
||
|
||
2. **Action:** I hover over the top-left corner where the GitLab logo is located.
|
||
- **Page Changes:** A dropdown menu appears with options including "Switch to," "Projects," "Groups," and "Explore."
|
||
- **Possible Purpose:** The purpose is to navigate to a different section or project within GitLab. Hovering reveals available navigation options for easier selection.
|
||
|
||
3. **Action:** I move the cursor to the search bar at the top of the page and click inside it.
|
||
- **Page Changes:** The search bar becomes active, and a placeholder text "Search GitLab" is visible inside it.
|
||
- **Possible Purpose:** The intent is to initiate a search within GitLab, possibly to find specific projects, issues, or other resources quickly.
|
||
|
||
4. **Action:** I type the text "gitlab.com/ByteBlaze11/project.contributor.me" into the search bar.
|
||
- **Page Changes:** As I type, the search bar dynamically updates with suggestions based on the input text. Suggestions include links and sections that match the typed URL.
|
||
- **Possible Purpose:** The goal is to locate a specific GitLab project or repository using its URL for direct access or reference.
|
||
|
||
5. **Action:** I use the keyboard to press the down arrow key.
|
||
- **Page Changes:** The focus moves to the first suggestion in the dropdown list under the search bar.
|
||
- **Possible Purpose:** The intent is to select one of the suggested search results without using the mouse, streamlining the navigation process.
|
||
|
||
6. **Action:** I press the enter key on the keyboard.
|
||
- **Page Changes:** The webpage navigates to the selected suggestion from the dropdown list, which is a link related to the typed URL.
|
||
- **Possible Purpose:** The action confirms the selection of the desired search result, leading to the specific GitLab project or page associated with the URL.
|
||
|
||
### Summary:
|
||
In this video segment, I navigate through the GitLab interface by accessing the "Monitor" section, utilizing the search functionality to find a specific project via its URL, and selecting a search result to view the project details. Each step is methodically executed to efficiently locate and access the required GitLab resources.
|
||
|
||
---
|
||
|
||
## Part 6
|
||
### Part 1: Initial State and First Action
|
||
|
||
#### Initial State:
|
||
- The video begins with a GitLab project page open, specifically on the "Issues" section of a project named "TODOTODO."
|
||
- The main content area displays a prompt encouraging the use of issues for collaboration, problem-solving, and planning work.
|
||
- On the left sidebar, various navigation options are visible, including "Project information," "Repository," "Issues," "Merge requests," etc.
|
||
- The top navigation bar includes a search bar with the placeholder text "Search GitLab."
|
||
|
||
#### Action:
|
||
- I click on the search bar at the top of the page.
|
||
|
||
#### Page Changes:
|
||
- The search bar becomes active, allowing text input.
|
||
|
||
#### Possible Purpose:
|
||
- The likely intent is to initiate a search within GitLab, possibly to find specific issues, projects, or other elements related to the current project or across GitLab.
|
||
|
||
---
|
||
|
||
### Second Action
|
||
|
||
#### Action:
|
||
- I type the text `get_by_placeholder("Search GitLab")` into the search bar.
|
||
|
||
#### Page Changes:
|
||
- No immediate visual changes occur as the text is being typed. The search has not been executed yet.
|
||
|
||
#### Possible Purpose:
|
||
- This action suggests an intention to either test or demonstrate a method for interacting with the search bar programmatically, rather than performing an actual search. The text entered does not correspond to a typical search query but appears to be a command or function call.
|
||
|
||
---
|
||
|
||
### Third Action
|
||
|
||
#### Action:
|
||
- I navigate away from the search bar and instead click on the "Byte Blazea11yproject.contributor.me" link in the breadcrumb navigation at the top of the page.
|
||
|
||
#### Page Changes:
|
||
- The page transitions from the "Issues" section of the "TODOTODO" project to the profile page of the user "Byte Blazea11yproject.contributor.me."
|
||
- The new page displays the user's profile information, including their username, bio, location, and activity heatmap.
|
||
|
||
#### Possible Purpose:
|
||
- The purpose of this action is to view the details of the user associated with the current project or issue. This could be to gather more information about the user's contributions, activity, or settings relevant to the project.
|
||
|
||
---
|
||
|
||
### Fourth Action
|
||
|
||
#### Action:
|
||
- I scroll down slightly on the user profile page.
|
||
|
||
#### Page Changes:
|
||
- The visible content on the page shifts downward, revealing more of the user's activity heatmap and additional sections of their profile.
|
||
|
||
#### Possible Purpose:
|
||
- Scrolling allows for a broader view of the user's activity and contributions over time, providing context about their engagement level and areas of focus within GitLab.
|
||
|
||
---
|
||
|
||
### Fifth Action
|
||
|
||
#### Action:
|
||
- I click on the "Issues" tab located under the user's profile overview.
|
||
|
||
#### Page Changes:
|
||
- The page transitions to display a list of issues associated with the user "Byte Blazea11yproject.contributor.me."
|
||
- The list includes various issues with titles, statuses, labels, and timestamps.
|
||
|
||
#### Possible Purpose:
|
||
- The intent is to review the issues that the user has created, assigned, or is involved in. This provides insight into the user's role and responsibilities within projects they contribute to.
|
||
|
||
---
|
||
|
||
### Summary of Actions in Part 1:
|
||
1. **Initial State**: On the "Issues" section of the "TODOTODO" project.
|
||
2. **First Action**: Click on the search bar at the top.
|
||
3. **Second Action**: Type `get_by_placeholder("Search GitLab")` into the search bar.
|
||
4. **Third Action**: Click on the "Byte Blazea11yproject.contributor.me" link in the breadcrumb navigation.
|
||
5. **Fourth Action**: Scroll down slightly on the user profile page.
|
||
6. **Fifth Action**: Click on the "Issues" tab under the user's profile overview.
|
||
|
||
Each step logically follows from the previous one, focusing on navigating through GitLab to gather information about a specific user and their involvement in issues.
|
||
|
||
---
|
||
|
||
## Part 7
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Initial State**
|
||
- **Action:** The video begins with the GitLab "Issues" page open, displaying a list of issues filtered by the assignee "Byte Blazea11yproject.contributor.me."
|
||
- **Page Changes:** No action is taken yet, so the page remains static, showing 13 open issues, 53 closed issues, and 66 issues in total.
|
||
- **Possible Purpose:** The initial state sets the context for the subsequent actions, indicating that the user is about to interact with the issues assigned to a specific contributor.
|
||
|
||
#### 2. **Scrolling Through the Issues List**
|
||
- **Action:** I scroll down the page.
|
||
- **Page Changes:** The list of issues moves upward, revealing more issues further down the list.
|
||
- **Possible Purpose:** The purpose of scrolling is to view additional issues that are not initially visible on the screen, allowing for a comprehensive review of all assigned tasks.
|
||
|
||
#### 3. **Hovering Over an Issue**
|
||
- **Action:** I hover over the issue titled "[Feature suggestion] Support linking to an accessibility statement."
|
||
- **Page Changes:** Hovering over the issue highlights it slightly, making it stand out from the other issues.
|
||
- **Possible Purpose:** Hovering over the issue likely serves to focus attention on this specific task, possibly in preparation for selecting or interacting with it.
|
||
|
||
#### 4. **Clicking on an Issue**
|
||
- **Action:** I click on the issue titled "[Feature suggestion] Support linking to an accessibility statement."
|
||
- **Page Changes:** The page transitions to the detailed view of the selected issue, displaying more information such as the description, comments, and related details.
|
||
- **Possible Purpose:** The intent of clicking on the issue is to access its detailed information, which may include reviewing the issue's specifics, adding comments, or taking further action such as assigning it to someone else or changing its status.
|
||
|
||
### Summary
|
||
In this video segment, the actions performed include:
|
||
1. Observing the initial state of the "Issues" page.
|
||
2. Scrolling through the list of issues to view more content.
|
||
3. Hovering over a specific issue to highlight it.
|
||
4. Clicking on the highlighted issue to view its detailed information.
|
||
|
||
Each step is executed with the apparent purpose of navigating and interacting with the issues assigned to a particular contributor, focusing on one specific issue for detailed review.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "action_0",
|
||
"idx": 0,
|
||
"action_repr": "frame.clicklocator(\"[id=\\\"__BVID__10__BV_toggle_\\\"]\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_1",
|
||
"idx": 1,
|
||
"action_repr": "frame.clicklocator(\"[id=\\\"__BVID__10__BV_toggle_\\\"]\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "user-menu",
|
||
"idx": 4,
|
||
"action_repr": "frame.clickget_by_test_id(\"user-menu\").get_by_role(\"link\", name=\"Byte Blazea11yproject.contributor.me\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "text_Projects New project",
|
||
"idx": 3,
|
||
"action_repr": "frame.clickget_by_text(\"Projects New project\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "user-profile-link",
|
||
"idx": 5,
|
||
"action_repr": "frame.clickget_by_test_id(\"user-profile-link\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Personal projects",
|
||
"idx": 6,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Personal projects\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_7",
|
||
"idx": 7,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search GitLab\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_8",
|
||
"idx": 8,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search GitLab\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_9",
|
||
"idx": 9,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search GitLab\")ArrowLeft",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_10",
|
||
"idx": 10,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search GitLab\")ArrowLeft",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_11",
|
||
"idx": 11,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search GitLab\")ArrowLeft",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_12",
|
||
"idx": 12,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search GitLab\")ArrowLeft",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/users/byteblaze/projects"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_13",
|
||
"idx": 13,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search GitLab\")Enter",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=my%20todo&nav_source=navbar"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=my%20todo&nav_source=navbar"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Byte Blazea11yproject.contributor.me\\n/ TODO",
|
||
"idx": 14,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Byte Blazea11yproject.contributor.me\\n/ TODO\", exact=True)",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todo"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todo"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "text_Show your appreciation to those who have contributed to the project.",
|
||
"idx": 15,
|
||
"action_repr": "frame.clickget_by_text(\"Show your appreciation to those who have contributed to the project.\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todo"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todo"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Byte Blazea11yproject.contributor.me\\n/ TODOTODO",
|
||
"idx": 16,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Byte Blazea11yproject.contributor.me\\n/ TODOTODO\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=my%20todo&nav_source=navbar"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=my%20todo&nav_source=navbar"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_17",
|
||
"idx": 17,
|
||
"action_repr": "frame.clicklocator(\"[id=\\\"__BVID__104__BV_toggle_\\\"]\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_18",
|
||
"idx": 18,
|
||
"action_repr": "frame.clicklocator(\"[id=\\\"__BVID__104__BV_toggle_\\\"]\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_19",
|
||
"idx": 19,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search GitLab\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Issues assigned to me",
|
||
"idx": 23,
|
||
"action_repr": "frame.clickget_by_label(\"Issues assigned to me\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo/-/issues/?assignee_username=byteblaze"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "breadcrumb-links",
|
||
"idx": 21,
|
||
"action_repr": "frame.clickget_by_test_id(\"breadcrumb-links\").get_by_role(\"link\", name=\"Byte Blazea11yproject.contributor.me\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo/-/issues/?assignee_username=byteblaze"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze/todotodo/-/issues/?assignee_username=byteblaze"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_22",
|
||
"idx": 22,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search GitLab\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/byteblaze"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |