283 lines
13 KiB
Plaintext
283 lines
13 KiB
Plaintext
# Instruction
|
||
- You are an expert in cleaning process data descriptions. Given a task, you are provided with a set of annotation description
|
||
data for a certain visual LLM related to human user operation videos. Plus, You are provided with full trace of playwright action,
|
||
whic includes action and url before and after the action.
|
||
- You need to analyze all the descriptive data and ultimately summarize a complete and reasonable user operation description that can accomplish the given task.
|
||
- For each strategy, give a clear list of the low level action sequence.
|
||
|
||
# Task
|
||
Today is 3/15/2023, generate a sales order report for last month
|
||
|
||
# Annotation description
|
||
## Part 1
|
||
### Step-by-Step Actions in the Provided Video Segment
|
||
|
||
#### 1. **Initial State: Dashboard Page**
|
||
- **Action:** The video starts with the Magento dashboard page visible.
|
||
- **Page Changes:** No changes occur as this is the starting point.
|
||
- **Possible Purpose:** The initial state sets the context for the subsequent actions.
|
||
|
||
#### 2. **Navigating to Reports Section**
|
||
- **Action:** I click on the "Reports" menu item in the left sidebar.
|
||
- **Page Changes:** The page transitions from the Dashboard to the Reports overview page.
|
||
- **Possible Purpose:** To access various report options available within the Magento system.
|
||
|
||
#### 3. **Selecting Orders Report**
|
||
- **Action:** On the Reports page, I click on the "Orders" link under the "Sales" section.
|
||
- **Page Changes:** The page navigates to the "Orders Report" page.
|
||
- **Possible Purpose:** To view and analyze order-related data and statistics.
|
||
|
||
#### 4. **Interacting with Date Used Filter**
|
||
- **Action:** I click on the "Date Used" dropdown menu.
|
||
- **Page Changes:** A dropdown list appears with options such as "Order Created," "Invoiced," "Shipped," etc.
|
||
- **Possible Purpose:** To specify the type of date to be used for filtering the orders report.
|
||
|
||
#### 5. **Selecting "Order Created" Option**
|
||
- **Action:** I select the "Order Created" option from the dropdown menu.
|
||
- **Page Changes:** The "Order Created" option is highlighted, indicating it has been selected.
|
||
- **Possible Purpose:** To filter the orders based on the date they were created.
|
||
|
||
#### 6. **Interacting with Date Range Filters**
|
||
- **Action:** I click on the "From" date input field.
|
||
- **Page Changes:** A date picker interface appears, allowing me to select a specific start date.
|
||
- **Possible Purpose:** To set a starting date for the order report filter.
|
||
|
||
- **Action:** I click on the "To" date input field.
|
||
- **Page Changes:** Another date picker interface appears, allowing me to select an end date.
|
||
- **Possible Purpose:** To set an ending date for the order report filter.
|
||
|
||
#### 7. **Setting Order Status Filter**
|
||
- **Action:** I click on the "Order Status" dropdown menu.
|
||
- **Page Changes:** A dropdown list appears with various order status options.
|
||
- **Possible Purpose:** To choose specific order statuses to include in the report.
|
||
|
||
- **Action:** I select an option from the "Order Status" dropdown (the exact option is not specified in the video).
|
||
- **Page Changes:** The selected order status is applied to the filter criteria.
|
||
- **Possible Purpose:** To refine the report to include only orders with the selected status.
|
||
|
||
#### 8. **Showing Actual Values Option**
|
||
- **Action:** I click on the "Show Actual Values" dropdown menu.
|
||
- **Page Changes:** A dropdown list appears with options like "Yes" and "No."
|
||
- **Possible Purpose:** To decide whether to display actual values in the report.
|
||
|
||
- **Action:** I select the "No" option from the "Show Actual Values" dropdown.
|
||
- **Page Changes:** The "No" option is applied, indicating that actual values will not be shown.
|
||
- **Possible Purpose:** To configure the report settings according to preference or requirement.
|
||
|
||
#### 9. **Submitting the Filter Criteria**
|
||
- **Action:** I click on the "Show Report" button.
|
||
- **Page Changes:** The page processes the filter criteria and displays the filtered orders report.
|
||
- **Possible Purpose:** To generate and view the orders report based on the specified filters.
|
||
|
||
### Conclusion
|
||
The video segment demonstrates a series of deliberate actions aimed at navigating to the Orders Report page, configuring filter settings, and generating a customized report. Each step is methodically executed to achieve the intended result of viewing specific order data within defined parameters.
|
||
|
||
---
|
||
|
||
## Part 2
|
||
### Step-by-Step Actions in the Video Segment
|
||
|
||
#### 1. **Opening the Date Picker for "From" Field**
|
||
- **Action:** I click on the calendar icon next to the "From" date field.
|
||
- **Page Changes:** A date picker modal appears, allowing me to select a specific date.
|
||
- **Possible Purpose:** The likely intent is to choose a starting date for the order report filter.
|
||
|
||
#### 2. **Selecting a Date in the Date Picker**
|
||
- **Action:** I click on a specific date within the date picker (April 15, 2023).
|
||
- **Page Changes:** The selected date (04/15/2023) is populated in the "From" date field.
|
||
- **Possible Purpose:** To set the beginning of the date range for the order report.
|
||
|
||
#### 3. **Opening the Date Picker for "To" Field**
|
||
- **Action:** I click on the calendar icon next to the "To" date field.
|
||
- **Page Changes:** Another date picker modal appears, enabling the selection of an end date.
|
||
- **Possible Purpose:** To choose an ending date for the order report filter.
|
||
|
||
#### 4. **Selecting a Date in the "To" Date Picker**
|
||
- **Action:** I click on a specific date within the date picker (May 15, 2023).
|
||
- **Page Changes:** The selected date (05/15/2023) is populated in the "To" date field.
|
||
- **Possible Purpose:** To finalize the date range for the order report, ensuring it covers the desired period.
|
||
|
||
#### 5. **Clicking "Show Report" Button**
|
||
- **Action:** I click the "Show Report" button located at the top right of the page.
|
||
- **Page Changes:** The page processes the request and displays an orders report table with data corresponding to the selected date range.
|
||
- **Possible Purpose:** To generate and view the orders report based on the specified filters.
|
||
|
||
#### 6. **Reviewing the Generated Report**
|
||
- **Action:** I observe the displayed orders report.
|
||
- **Page Changes:** The report table shows various columns such as Interval, Orders, Sales Items, Sales Total, etc., with data entries for each day within the selected date range.
|
||
- **Possible Purpose:** To analyze the order data for the specified period, possibly for business insights or record-keeping.
|
||
|
||
### Summary
|
||
In this video segment, I interact with the "Orders Report" page by setting a date range using the "From" and "To" date pickers, then generating and reviewing the report by clicking the "Show Report" button. Each action is methodically performed to filter and display relevant order data for analysis.
|
||
|
||
# Playwright action
|
||
[
|
||
{
|
||
"action_uid": "link_\ue60a Reports",
|
||
"idx": 0,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"\ue60a Reports\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_Orders",
|
||
"idx": 1,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"Orders\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/admin/dashboard/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_From",
|
||
"idx": 2,
|
||
"action_repr": "frame.clickget_by_label(\"From\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_3",
|
||
"idx": 3,
|
||
"action_repr": "frame.clicklocator(\"#sales_report_base_fieldset div\").filter(has_text=\"From undefined\").get_by_role(\"button\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "label_Select month",
|
||
"idx": 7,
|
||
"action_repr": "frame.selectOptionget_by_label(\"Select month\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "link_15",
|
||
"idx": 8,
|
||
"action_repr": "frame.clickget_by_role(\"link\", name=\"15\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "action_6",
|
||
"idx": 6,
|
||
"action_repr": "frame.clicklocator(\"#sales_report_base_fieldset div\").filter(has_text=\"To undefined\").get_by_role(\"button\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/"
|
||
}
|
||
},
|
||
{
|
||
"action_uid": "button_Show Report",
|
||
"idx": 9,
|
||
"action_repr": "frame.clickget_by_role(\"button\", name=\"Show Report\")",
|
||
"before": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/filter/cmVwb3J0X3R5cGU9Y3JlYXRlZF9hdF9vcmRlciZwZXJpb2RfdHlwZT1kYXkmZnJvbT0wNCUyRjE1JTJGMjAyMyZ0bz0wNSUyRjE1JTJGMjAyMyZzaG93X29yZGVyX3N0YXR1c2VzPTAmc2hvd19lbXB0eV9yb3dzPTAmc2hvd19hY3R1YWxfY29sdW1ucz0w/"
|
||
},
|
||
"after": {
|
||
"url": "http://ec2-3-135-39-80.us-east-2.compute.amazonaws.com:7780/admin/reports/report_sales/sales/filter/cmVwb3J0X3R5cGU9Y3JlYXRlZF9hdF9vcmRlciZwZXJpb2RfdHlwZT1kYXkmZnJvbT0wNCUyRjE1JTJGMjAyMyZ0bz0wNSUyRjE1JTJGMjAyMyZzaG93X29yZGVyX3N0YXR1c2VzPTAmc2hvd19lbXB0eV9yb3dzPTAmc2hvd19hY3R1YWxfY29sdW1ucz0w/"
|
||
}
|
||
}
|
||
]
|
||
|
||
# Output format
|
||
- 先总结整个任务的Objective，然后按照Strategy-SubStrategy-action三级层次来给出整个过程，
|
||
- 接着给出整个操作流程后的观察和有趣的发现，最后严格按照json格式输出三级层次的过程描述。
|
||
- 最后的输出json应该是包在```{json}```之间，最底层动作需要包含描述、对应的playwright动作指令顺序编号，以及具体指令内容。
|
||
|
||
# Example
|
||
### Complete User Operation Description to Display Labeled Issues in kkroening/ffmpeg-python
|
||
|
||
**Objective:** Filter and display all issues labeled as "question" in the kkroening/ffmpeg-python repository.
|
||
|
||
---
|
||
|
||
#### **Strategy 1: Navigate to the Repository**
|
||
**Low-Level Action Sequence:**
|
||
1. **Search for the user "kkroening"**
|
||
- Click the global search bar (placeholder: "Search GitLab").
|
||
- Type "kkroening" and press `Enter`.
|
||
2. **Select the user from results**
|
||
- Click the "Users" tab in search results.
|
||
- Click on "Karl Kroening @kkroening" in the user list.
|
||
3. **Access the repository**
|
||
- Navigate to the "Personal projects" section.
|
||
- Click on the "ffmpeg-python" project.
|
||
|
||
---
|
||
|
||
#### **Strategy 2: Filter Issues by Label**
|
||
**Low-Level Action Sequence:**
|
||
1. **Open the Issues tab**
|
||
- Scroll to the left sidebar menu.
|
||
- Click the "Issues" tab (displaying the count, e.g., "Issues 402").
|
||
2. **Apply label filtering**
|
||
- Click the search/filter bar in the issues list.
|
||
- Select the "Label" dropdown from the filter options.
|
||
- Type or select "question" from the label dropdown.
|
||
- Click the search/apply button to confirm the filter.
|
||
|
||
---
|
||
|
||
#### **Final Oberservation**
|
||
The issues list will refresh to show only issues with the "question" label. The URL will reflect the filter:
|
||
`.../ffmpeg-python/-/issues/?label_name[]=question`.
|
||
|
||
---
|
||
|
||
### Key Observations from Playwright Trace
|
||
- The final URL after filtering:
|
||
`http://ec2-3-135-39-80.../ffmpeg-python/-/issues/?label_name%5B%5D=question`
|
||
confirms the "question" label filter is applied.
|
||
- Critical interactions include selecting the "Label" dropdown and explicitly choosing "question" to refine results.
|
||
|
||
### Final output
|
||
```json
|
||
[{
|
||
"strategy" : "Navigate to the Repository",
|
||
"substrategies": [
|
||
{
|
||
"substrategy": "Search for the user \"kkroening\"",
|
||
"actions" : [
|
||
{
|
||
"description": "Click the global search bar (placeholder: \"Search GitLab\"). ",
|
||
"playwright_idx" : 18,
|
||
"playwright_instruction" : "frame.pressget_by_placeholder(\"Search GitLab\")Enter"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"substrategy": "Select the user from results",
|
||
"actions" : [
|
||
]
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"strategy" : "Filter Issues by Label",
|
||
"substrategies" : [
|
||
]
|
||
}]
|
||
``` |