255 lines
14 KiB
Plaintext
255 lines
14 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
Show me the command to clone the best GAN python implementation with SSH.
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions in the Provided Video Segment
|
||
|
||
#### 1. **Action:** I click on the search bar at the top of the page.
|
||
- **Page Changes:** The search bar becomes active, and a dropdown menu appears below it, displaying recent searches or suggestions.
|
||
- **Possible Purpose:** The likely intent is to initiate a search within the GitLab platform, possibly to find specific projects, issues, or users.
|
||
|
||
#### 2. **Action:** I type "gan" into the search bar.
|
||
- **Page Changes:** As I type, the dropdown menu updates dynamically to show search suggestions related to the text "gan." These suggestions include projects, users, and other relevant items that contain the term "gan."
|
||
- **Possible Purpose:** The purpose is to narrow down the search results to items specifically related to "gan," which could refer to "Generative Adversarial Networks" or another relevant term within the context of the platform.
|
||
|
||
#### 3. **Action:** I hover over the search suggestion for "Learn Anything / learn-anything."
|
||
- **Page Changes:** Hovering over this suggestion highlights it, making it visually distinct from other options in the dropdown menu.
|
||
- **Possible Purpose:** This action suggests an interest in exploring the "Learn Anything / learn-anything" project further, possibly to access its contents or details.
|
||
|
||
#### 4. **Action:** I click on the "Learn Anything / learn-anything" suggestion in the dropdown menu.
|
||
- **Page Changes:** The page transitions to display the details of the "Learn Anything / learn-anything" project. This new page shows information such as the project description, files, commits, and other relevant details.
|
||
- **Possible Purpose:** The intent is to view and potentially interact with the "Learn Anything / learn-anything" project, which may contain resources or information of interest.
|
||
|
||
### Summary
|
||
In this video segment, I initiate a search within the GitLab platform by clicking on the search bar and typing "gan." I then select the "Learn Anything / learn-anything" project from the search suggestions, leading to a detailed view of that specific project. Each step is focused on navigating and accessing relevant content within the platform.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the search bar labeled "What are you searching for?".
|
||
- **Page Changes:** The cursor is now active in the search bar, allowing me to type.
|
||
- **Possible Purpose:** My intention is to initiate a search query within the platform.
|
||
|
||
2. **Action:** I type "gan" into the search bar.
|
||
- **Page Changes:** As I type, a dropdown menu appears with search suggestions and results that match the keyword "gan".
|
||
- **Possible Purpose:** I am looking for projects, issues, or other elements related to "gan" (likely an abbreviation or term relevant to my interests).
|
||
|
||
3. **Action:** I select the first search result titled "Erik Linder-Norén / PyTorch-GAN" from the dropdown menu.
|
||
- **Page Changes:** The webpage transitions to the project page for "PyTorch-GAN" by Erik Linder-Norén.
|
||
- **Possible Purpose:** I intend to view the details of this specific project, possibly because it is relevant to my search for "gan".
|
||
|
||
4. **Action:** I observe the notification at the top of the page stating, "You can’t push or pull repositories using SSH until you add an SSH key to your profile."
|
||
- **Page Changes:** No action is taken, but the notification is visible, indicating a restriction on certain actions.
|
||
- **Possible Purpose:** I am acknowledging the notification, which informs me about a requirement for future actions involving SSH.
|
||
|
||
5. **Action:** I hover over the "Clone" button on the project page.
|
||
- **Page Changes:** A tooltip or additional information may appear, providing details about the cloning options.
|
||
- **Possible Purpose:** I am considering cloning the repository, and I am checking the available options or instructions for doing so.
|
||
|
||
6. **Action:** I click on the "Clone" button.
|
||
- **Page Changes:** A dropdown menu appears, offering different methods to clone the repository (e.g., HTTPS, SSH).
|
||
- **Possible Purpose:** I am preparing to clone the repository using one of the provided methods.
|
||
|
||
7. **Action:** I select the HTTPS option from the clone dropdown menu.
|
||
- **Page Changes:** The HTTPS URL for cloning the repository is displayed, ready to be copied.
|
||
- **Possible Purpose:** I have chosen to clone the repository using the HTTPS method, likely because it is a common and straightforward approach.
|
||
|
||
### Summary:
|
||
In this video segment, I perform a series of actions focused on searching for and accessing a specific project related to "gan". I navigate through the search functionality, select a relevant project, and prepare to clone its repository using the HTTPS method. Throughout these steps, I also acknowledge a notification about SSH key requirements, which may impact future actions involving the repository. Each action is driven by the goal of obtaining and potentially working with the code or resources provided in the "PyTorch-GAN" project.
|
||
|
||
---
|
||
|
||
## Part 3
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I click on the "Clone" button located on the right side of the webpage, near the top-right corner under the project title "PyTorch-GAN."
|
||
- **Page Changes:** A dropdown menu appears below the "Clone" button, displaying options for cloning the repository with SSH and HTTP, along with links to open the repository in various IDEs (Visual Studio Code, IntelliJ IDEA).
|
||
- **Possible Purpose:** The likely intent is to obtain the necessary URL or instructions to clone the repository to a local machine or an integrated development environment.
|
||
|
||
2. **Action:** I hover over the "Clone with SSH" section within the dropdown menu.
|
||
- **Page Changes:** The SSH URL (`ssh://git@metis.lti.cs.cmu.edu:8022/`) is highlighted, indicating it is ready to be copied.
|
||
- **Possible Purpose:** This action suggests preparation to copy the SSH URL, which is typically used for secure and authenticated access to the repository.
|
||
|
||
3. **Action:** I click on the copy icon next to the "Clone with SSH" URL.
|
||
- **Page Changes:** There is no visible change on the page, but the URL is copied to the clipboard (as indicated by standard browser behavior for copy actions).
|
||
- **Possible Purpose:** The purpose is to copy the SSH URL to the clipboard so it can be pasted into a terminal or command line interface for cloning the repository.
|
||
|
||
4. **Action:** I move the cursor away from the "Clone" dropdown menu.
|
||
- **Page Changes:** The dropdown menu closes, returning the view to the main project page.
|
||
- **Possible Purpose:** This action completes the task of obtaining the SSH URL and prepares for the next step, which might involve navigating to a terminal or command line interface to perform the actual cloning.
|
||
|
||
### Summary:
|
||
In this video segment, I interact with the "Clone" button to access and copy the SSH URL for the "PyTorch-GAN" repository. The actions are focused on preparing to clone the repository securely using SSH, as indicated by the sequence of opening the clone options, selecting the SSH method, and copying the URL.
|
||
|
||
---
|
||
|
||
## Part 4
|
||
### Step-by-Step Actions:
|
||
|
||
1. **Action:** I hover over the "Clone" button located on the right side of the webpage.
|
||
- **Page Changes:** No immediate changes occur on the page; however, a dropdown menu appears below the "Clone" button, revealing additional options for cloning the repository.
|
||
- **Possible Purpose:** The likely intent is to explore different methods available for cloning the repository, such as using SSH, HTTP, or opening it in an IDE.
|
||
|
||
2. **Action:** I move the cursor over the "Clone with SSH" option within the dropdown menu.
|
||
- **Page Changes:** The "Clone with SSH" option becomes highlighted, indicating it is the currently selected choice.
|
||
- **Possible Purpose:** This action suggests an intention to select the SSH method for cloning the repository, possibly due to its secure nature or personal preference.
|
||
|
||
3. **Action:** I click on the "Clone with SSH" option.
|
||
- **Page Changes:** The SSH URL for cloning the repository is displayed in a text box next to the "Clone with SSH" label. Additionally, a small icon resembling a clipboard appears next to the URL.
|
||
- **Possible Purpose:** The purpose is to obtain the SSH URL needed to clone the repository securely using SSH. The appearance of the clipboard icon suggests that the next step might involve copying this URL.
|
||
|
||
4. **Action:** I move the cursor towards the clipboard icon next to the SSH URL.
|
||
- **Page Changes:** The cursor changes to a hand icon, indicating that the clipboard icon is clickable and will perform an action (likely copying the URL).
|
||
- **Possible Purpose:** The intention is to copy the SSH URL to the clipboard, preparing to use it in a terminal or command line interface to clone the repository.
|
||
|
||
### Summary:
|
||
In this video segment, I interact with the "Clone" button to explore cloning options, specifically selecting the "Clone with SSH" method. I then prepare to copy the provided SSH URL, likely intending to use it for cloning the repository via a terminal or command line interface. Each action is focused on obtaining the necessary URL for secure repository cloning.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "action_0",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_placeholder(\"Search GitLab\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_1",
|
||
"idx": 1,
|
||
"action_repr": "frame.pressget_by_placeholder(\"Search GitLab\")Enter",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Erik Linder-Nor\u00e9n\\n/ PyTorch-GAN",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Erik Linder-Nor\u00e9n\\n/ PyTorch-GAN\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=gan&nav_source=navbar"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=gan&nav_source=navbar"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Clone",
|
||
"idx": 5,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Clone\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/search?search=gan&nav_source=navbar"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/eriklindernoren/PyTorch-GAN"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Copy URL",
|
||
"idx": 4,
|
||
"action_repr": "frame.clicklocator(\"li\").filter(has_text=\"Clone with SSH\").get_by_label(\"Copy URL\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/eriklindernoren/PyTorch-GAN"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:8023/eriklindernoren/PyTorch-GAN"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |