Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Executed a statement , but no operation was performed #141

Open
Rx2DD opened this issue Dec 5, 2024 · 1 comment
Open

Executed a statement , but no operation was performed #141

Rx2DD opened this issue Dec 5, 2024 · 1 comment

Comments

@Rx2DD
Copy link

Rx2DD commented Dec 5, 2024

Hi, I executed a statement, but no operation was performed. The log is as follows. Please help me check what the problem is.

PS D:\company\UFO-main> python -m ufo --task safetest

Welcome to use UFO??, A UI-focused Agent for Windows OS Interaction.


| | | || | / _
| | | || |
| | | |
| |
| || | | || |
_
/ |_| __/

Please enter your request to be completed??:
open chrome and access the google search page
Round 1, Step 1, HostAgent: Analyzing the user intent and decomposing the request...
Observations??: I observe that the current screenshot shows the Chrome browser is visible, and the Google search page is opened.
Thoughts??: The user request can be solely complete on the Chrome browser. I need to open the Chrome browser to access the Google search page.Plans??: (1) Open the Chrome browser to access the Google search page.
Next Selected application??: Chrome
Messages to AppAgent??: (1) You need to open the Chrome browser to access the Google search page.
(2) You can use the 'Start' menu to open the Chrome browser.
(3) Alternatively, you can press the 'Ctrl + Shift + J' key combination to open the Chrome browser directly.
Status??: CONTINUE
Comment??: I plan to open the Chrome browser to access the Google search page.
Creating an experience indexer...

No sentence-transformers model found with name sentence-transformers/all-mpnet-base-v2. Creating a new one with MEAN pooling.
Warning: Failed to load experience indexer from vectordb/experience/experience_db.
Creating an demonstration indexer...
No sentence-transformers model found with name sentence-transformers/all-mpnet-base-v2. Creating a new one with MEAN pooling.
Warning: Failed to load demonstration indexer from vectordb/demonstration/demonstration_db.

Round 1, Step 2, AppAgent: Completing the subtask [Open the Chrome browser to access the Google search page.] on application [Chrome].
Error Occurs at AppAgentProcessor
Traceback (most recent call last):
File "D:\company\UFO-main\ufo\agents\processors\app_agent_processor.py", line 264, in parse_response
self.response_json = self.app_agent.response_to_dict(self.response)
File "D:\company\UFO-main\ufo\agents\agent\basic.py", line 162, in response_to_dict
return utils.json_parser(response)
File "D:\company\UFO-main\ufo\utils_init
.py", line 76, in json_parser
return json.loads(json_string)
File "D:\Programs\Python\Python310\lib\json_init
.py", line 346, in loads
return _default_decoder.decode(s)
File "D:\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "D:\Programs\Python\Python310\lib\json\decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 765 (char 764)

{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}]}

{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}], "Result": [], "UserConfirm": null}
{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}], "Result": [], "UserConfirm": null}
{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}], "Result": [], "UserConfirm": null}
{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}], "Result": [], "UserConfirm": null}
{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}], "Result": [], "UserConfirm": null}
{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}], "Result": [], "UserConfirm": null}
{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for information on the internet.", "(4) The task is now complete.", "Comment": "By using the 'search_box' API, I have opened the Google search page, allowing the user to search for information on the internet.", "SaveScreenshot": {"save": false, "reason": ""}], "Result": [], "UserConfirm": null}
{"Observation": "The screenshot shows that I am on the Chrome browser. The Google search page is visible in the screenshot. The last action took effect by opening the Chrome browser to access the Google search page.", "Thought": "Based on the screenshots and the control item list, I need to use the 'search_box' API to open the Google search page. This API is suitable for searching for information on the internet.", "ControlLabel": "search_box", "ControlText": "Google Search", "Function": "search_box", "Args": {}, "Status": "CONTINUE", "Plan": ["(1) Call the 'search_box' API to open the Google search page.", "(2) The Google search page is now open.", "(3) The user can now search for
Observations??: None
Thoughts??: None
Selected item???: [No control selected.], Label: [No control label sel
ected.]
Action applied??: None()
Status??: None
Next Plan??:
Comment??: None
Please enter your new request. Enter 'N' for exit.

@Rx2DD Rx2DD changed the title executed a statement , but no operation was performed Executed a statement , but no operation was performed Dec 5, 2024
@vyokky
Copy link
Contributor

vyokky commented Dec 16, 2024

Which model do you used? It seems that the model failed to produce a json format output, so its action cannot be parsed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants