Merge pull request #39 from AkiChase/dev

Scrcpy Mask v0.4.3

README-zh.md

@@ -19,7 +19,7 @@
 - [x] 可视化编辑按键映射配置
 - [x] 按键映射配置的导入与导出
 - [x] 更新检查
-- [x] 在按键映射和插入文本之间切换
+- [x] 在按键映射和按键输入之间切换
 - [x] 国际化
 - [ ] 手柄按键映射
 - [ ] 更好的宏
@@ -72,6 +72,29 @@
 6. 导航到键映射页面，切换或编辑键映射配置。
 7. 返回到蒙版界面，开始使用吧！
 
+## 关于宏
+
+目前宏的结构仅仅是一个 JSON 对象，功能有限，仅仅是作为过渡使用的。请勿投入太多时间来编写宏，因为**宏的编写规范随时可能因版本更新而变动**。
+
+宏的示例可见 [hotkey.ts](https://github.com/AkiChase/scrcpy-mask/blob/master/src/hotkey.ts) 的 `async function execMacro` 函数注释。
+
+比如 `key-input-mode` 宏，可以从按键映射模式切换到按键输入模式，常用于文本输入。示例如下：
+
+```json
+[{ "args": [], "type": "key-input-mode" }]
+```
+
+## 错误报告
+
+提问时请尽可能全面而清晰地提供问题相关的信息，包括操作系统和软件版本。特别是如果有错误输出，请务必附带相关日志。
+
+日志有两个来源，可能对定位并解决错误有所帮助。一般来说，Web 日志中就可以找到错误输出。
+
+1. Web 日志：通过 `Ctrl+Shift+I` 或 `Cmd+Opt+I` 打开开发者工具，点击控制台 (console)，查看控制台内输出的信息。
+2. Rust 日志：
+   1. 在 macOS 或 Linux 系统下，可以进入安装位置，使用**终端**运行 `scrcpy-mask`，可在终端中实时看到程序的输出信息。
+   2. 在 Windows 系统下，目前只能克隆项目后自行运行，查看 Rust 输出信息。
+
 ## 贡献
 
 如果你对这个项目感兴趣，欢迎提 PR 或 Issue。但我的时间和精力有限，所以可能无法全部及时处理。

README.md

@@ -21,7 +21,7 @@ Furthermore, to better support interaction between Scrcpy Mask and Android devic
 - [x] Visually setting the mapping
 - [x] Key mapping config import and export
 - [x] Update check
-- [x] Switch between key mapping and input-text box
+- [x] Toggle between key mapping and key input
 - [x] Internationalization (i18n)
 - [ ] Gamepad key mapping
 - [ ] Better macro support
@@ -30,7 +30,7 @@ Furthermore, to better support interaction between Scrcpy Mask and Android devic
 
 ## Demonstration video
 
-- [如何用电脑玩FPS手游？这样的“安卓模拟器”，也不是不可以-哔哩哔哩](https://www.bilibili.com/video/BV1EU411Z7TC/?share_source=copy_web&vd_source=36923115230d8a46ae8b587fc5348e6e)
+- [如何用电脑玩 FPS 手游？这样的“安卓模拟器”，也不是不可以-哔哩哔哩](https://www.bilibili.com/video/BV1EU411Z7TC/?share_source=copy_web&vd_source=36923115230d8a46ae8b587fc5348e6e)
 - [M 系列 Mac 电脑玩王者，暃排位实录，使用 Android Stuido 模拟器和开源 Scrcpy Mask 按键映射工具-哔哩哔哩](https://b23.tv/q6iDW1w)
 - [自制跨平台开源项目 Scrcpy Mask ，像模拟器一样用键鼠控制任意安卓设备！以 M 系列芯片 MacBook 打王者为例-哔哩哔哩](https://b23.tv/gqmriXr)
 - [如何用 PC 控制安卓手机打王者？只要思想不滑坡，办法总比困难多！-哔哩哔哩](https://b23.tv/dmUOpff)
@@ -53,7 +53,7 @@ Furthermore, to better support interaction between Scrcpy Mask and Android devic
 ![](https://pic.superbed.cc/item/66371911f989f2fb975b62a3.png)
 
 - Mask above game
- 
+
 ![](https://pic.superbed.cc/item/66373c8cf989f2fb97679dfd.png)
 
 ![](https://pic.superbed.cc/item/6649cf0cfcada11d37c05b5e.jpg)
@@ -68,12 +68,35 @@ Furthermore, to better support interaction between Scrcpy Mask and Android devic
    2. For emulator, you don't need screen mirror, and emulator generally default to enabling ADB wired debugging. So this is the best way for game, I think.
 3. Launch the software and navigate to the Device page.
    1. Find your device among the available devices (if not found, please search for how to enable ADB debugging for your device).
-   3. Right-click on your device again and choose "Control this device".
+   2. Right-click on your device again and choose "Control this device".
 4. Navigate to the Settings page -> Mask Settings, set the width and height of the mask to the same ratio of the device screen size and ensure that the mask size is appropriate.
 5. Navigate to the Mask page where you can see a transparent mask. Next, adjust and move your emulator window or screen mirroring window to align the displayed content area with the transparent mask area.
 6. Navigate to the Key mapping page and switch or edit the key mapping configs.
 7. Return to the Mask page and start enjoying.
 
+## About Macros
+
+Currently, the structure of macros is simply a JSON object with limited functionality, serving as a transitional solution. **Please refrain from investing too much time in writing macros, as the specifications for macro creation may change with version updates.**
+
+An example of macros can be found in the `async function execMacro` function in [hotkey.ts](https://github.com/AkiChase/scrcpy-mask/blob/master/src/hotkey.ts) file.
+
+For instance, the `key-input-mode` macro can switch from key mapping mode to key input mode, commonly used for text input. An example is as follows:
+
+```json
+[{ "args": [], "type": "key-input-mode" }]
+```
+
+## Error Report
+
+When asking a question, please provide as much detailed information as possible regarding the issue, including the operating system and software version. Specifically, if there is an error output, please be sure to include the relevant logs.
+
+There are two sources of logs that might help in identifying and solving the error. Generally, the error output can be found in the Web logs.
+
+1. Web Logs: Open Developer Tools by pressing `Ctrl+Shift+I` or `Cmd+Opt+I`, click on the console tab, and check the information output in the console.
+2. Rust Logs:
+   1. On macOS or Linux, navigate to the installation directory, use the **terminal** to run `scrcpy-mask`, and you can see the program's output in real-time in the terminal.
+   2. On Windows, you need to clone the project and run it yourself to view the Rust output.
+
 ## Contribution.
 
 If you are interested in this project, you are welcome to submit pull request or issue. But my time and energy is limited, so I may not be able to deal with it all.

package.json

@@ -1,7 +1,7 @@
 {
   "name": "scrcpy-mask",
   "private": true,
-  "version": "0.4.2",
+  "version": "0.4.3",
   "type": "module",
   "scripts": {
     "dev": "vite",

src-tauri/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "scrcpy-mask"
-version = "0.4.2"
+version = "0.4.3"
 description = "A Tauri App"
 authors = ["AkiChase"]
 edition = "2021"

src-tauri/resource/default-key-config.json

@@ -1,5 +1,5 @@
 [
-    {"list":[{"key":{"down":"KeyS","left":"KeyA","right":"KeyD","up":"KeyW"},"note":"方向轮盘","offset":175,"pointerId":1,"posX":183,"posY":566,"type":"SteeringWheel"},{"key":"KeyQ","note":"技能1","pointerId":2,"posX":951,"posY":636,"range":30,"type":"DirectionalSkill"},{"key":"KeyE","note":"技能2","pointerId":2,"posX":1025,"posY":500,"range":30,"type":"DirectionalSkill"},{"key":"AltLeft","note":"技能3","pointerId":2,"posX":1160,"posY":420,"range":30,"type":"DirectionalSkill"},{"key":"Space","note":"取消技能","pointerId":2,"posX":1160,"posY":140,"type":"CancelSkill"},{"key":"KeyB","note":"回城","pointerId":3,"posX":650,"posY":650,"time":80,"type":"Tap"},{"key":"KeyC","note":"回复","pointerId":3,"posX":740,"posY":650,"time":80,"type":"Tap"},{"key":"KeyF","note":"召唤师技能","pointerId":2,"posX":838,"posY":647,"range":50,"type":"DirectionalSkill"},{"key":"M2","note":"攻击","pointerId":3,"posX":1174,"posY":618,"time":80,"type":"Tap"},{"key":"Digit1","note":"技能1升级","pointerId":3,"posX":880,"posY":560,"time":80,"type":"Tap"},{"key":"Digit2","note":"技能2升级","pointerId":3,"posX":960,"posY":430,"time":80,"type":"Tap"},{"key":"Digit3","note":"技能3升级","pointerId":3,"posX":1090,"posY":350,"time":80,"type":"Tap"},{"key":"Digit5","note":"快速购买1","pointerId":3,"posX":133,"posY":289,"time":80,"type":"Tap"},{"key":"Digit6","note":"快速购买2","pointerId":3,"posX":130,"posY":370,"time":80,"type":"Tap"},{"key":"M3","note":"观察","pointerId":4,"posX":1000,"posY":200,"scale":1,"type":"Observation"},{"key":"Tab","macro":{"down":[{"args":["default",5,1185,40,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1220,100,80],"type":"touch"}]},"note":"战绩面板","posX":1185,"posY":40,"type":"Macro"},{"key":"ShiftLeft","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1200,60,80],"type":"touch"}]},"note":"商店","posX":44,"posY":302,"type":"Macro"},{"key":"KeyZ","macro":{"down":[{"args":["default",5,250,230,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,640,150,80],"type":"touch"}]},"note":"地图","posX":250,"posY":230,"type":"Macro"},{"key":"Backquote","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,510,630,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1165,575,80],"type":"touch"},{"args":["default",5,1200,65,80],"type":"touch"},{"args":[200],"type":"sleep"},{"args":["default",5,125,300,80],"type":"touch"}],"loop":null,"up":null},"note":"换装2","posX":236,"posY":66,"type":"Macro"},{"directional":false,"key":"ControlLeft","note":"","pointerId":2,"posX":839,"posY":647,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"directional":false,"key":"M4","note":"","pointerId":2,"posX":952,"posY":636,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"directional":false,"key":"WheelDown","note":"","pointerId":2,"posX":1155,"posY":280,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"key":"Enter","macro":{"down":[{"args":["default",5,1245,280,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1160,560,80],"type":"touch"},{"args":[1],"type":"input-text"}],"loop":null,"up":null},"note":"聊天","posX":1244,"posY":281,"type":"Macro"}],"relativeSize":{"h":720,"w":1280},"title":"AVD-王者荣耀-三技能-导入默认"},
+    {"list":[{"key":{"down":"KeyS","left":"KeyA","right":"KeyD","up":"KeyW"},"note":"方向轮盘","offset":175,"pointerId":1,"posX":183,"posY":566,"type":"SteeringWheel"},{"key":"KeyQ","note":"技能1","pointerId":2,"posX":951,"posY":636,"range":30,"type":"DirectionalSkill"},{"key":"KeyE","note":"技能2","pointerId":2,"posX":1025,"posY":500,"range":30,"type":"DirectionalSkill"},{"key":"AltLeft","note":"技能3","pointerId":2,"posX":1160,"posY":420,"range":30,"type":"DirectionalSkill"},{"key":"Space","note":"取消技能","pointerId":2,"posX":1160,"posY":140,"type":"CancelSkill"},{"key":"KeyB","note":"回城","pointerId":3,"posX":650,"posY":650,"time":80,"type":"Tap"},{"key":"KeyC","note":"回复","pointerId":3,"posX":740,"posY":650,"time":80,"type":"Tap"},{"key":"KeyF","note":"召唤师技能","pointerId":2,"posX":838,"posY":647,"range":50,"type":"DirectionalSkill"},{"key":"M2","note":"攻击","pointerId":3,"posX":1174,"posY":618,"time":80,"type":"Tap"},{"key":"Digit1","note":"技能1升级","pointerId":3,"posX":880,"posY":560,"time":80,"type":"Tap"},{"key":"Digit2","note":"技能2升级","pointerId":3,"posX":960,"posY":430,"time":80,"type":"Tap"},{"key":"Digit3","note":"技能3升级","pointerId":3,"posX":1090,"posY":350,"time":80,"type":"Tap"},{"key":"Digit5","note":"快速购买1","pointerId":3,"posX":133,"posY":289,"time":80,"type":"Tap"},{"key":"Digit6","note":"快速购买2","pointerId":3,"posX":130,"posY":370,"time":80,"type":"Tap"},{"key":"M3","note":"观察","pointerId":4,"posX":1000,"posY":200,"scale":1,"type":"Observation"},{"key":"Tab","macro":{"down":[{"args":["default",5,1185,40,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1220,100,80],"type":"touch"}]},"note":"战绩面板","posX":1185,"posY":40,"type":"Macro"},{"key":"ShiftLeft","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1200,60,80],"type":"touch"}]},"note":"商店","posX":44,"posY":302,"type":"Macro"},{"key":"KeyZ","macro":{"down":[{"args":["default",5,250,230,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,640,150,80],"type":"touch"}]},"note":"地图","posX":250,"posY":230,"type":"Macro"},{"key":"Backquote","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,510,630,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1165,575,80],"type":"touch"},{"args":["default",5,1200,65,80],"type":"touch"},{"args":[200],"type":"sleep"},{"args":["default",5,125,300,80],"type":"touch"}],"loop":null,"up":null},"note":"换装2","posX":236,"posY":66,"type":"Macro"},{"directional":false,"key":"ControlLeft","note":"","pointerId":2,"posX":839,"posY":647,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"directional":false,"key":"M4","note":"","pointerId":2,"posX":952,"posY":636,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"directional":false,"key":"WheelDown","note":"","pointerId":2,"posX":1155,"posY":280,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"key":"Enter","macro":{"down":[{"args":["default",5,1245,280,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1160,560,80],"type":"touch"},{"args":[],"type":"key-input-mode"}],"loop":null,"up":null},"note":"聊天","posX":1244,"posY":281,"type":"Macro"}],"relativeSize":{"h":720,"w":1280},"title":"AVD-王者荣耀-三技能-导入默认"},
-    {"list":[{"key":{"down":"KeyS","left":"KeyA","right":"KeyD","up":"KeyW"},"note":"方向轮盘","offset":175,"pointerId":1,"posX":183,"posY":566,"type":"SteeringWheel"},{"key":"KeyQ","note":"技能1","pointerId":2,"posX":952,"posY":636,"range":50,"type":"DirectionalSkill"},{"key":"AltLeft","note":"技能2","pointerId":2,"posX":979,"posY":526,"range":50,"type":"DirectionalSkill"},{"key":"KeyE","note":"技能3","pointerId":2,"posX":1074,"posY":438,"range":50,"type":"DirectionalSkill"},{"key":"Space","note":"取消技能","pointerId":2,"posX":1160,"posY":140,"type":"CancelSkill"},{"key":"KeyB","note":"回城","pointerId":3,"posX":644,"posY":650,"time":80,"type":"Tap"},{"key":"KeyC","note":"回复","pointerId":3,"posX":742,"posY":650,"time":80,"type":"Tap"},{"key":"KeyF","note":"召唤师技能","pointerId":2,"posX":838,"posY":647,"range":50,"type":"DirectionalSkill"},{"key":"M2","note":"攻击","pointerId":3,"posX":1179,"posY":621,"time":80,"type":"Tap"},{"key":"Digit1","note":"技能1升级","pointerId":3,"posX":895,"posY":564,"time":80,"type":"Tap"},{"key":"Digit2","note":"技能2升级","pointerId":3,"posX":922,"posY":456,"time":80,"type":"Tap"},{"key":"Digit3","note":"技能3升级","pointerId":3,"posX":1015,"posY":376,"time":80,"type":"Tap"},{"key":"Digit5","note":"快速购买1","pointerId":3,"posX":133,"posY":289,"time":80,"type":"Tap"},{"key":"Digit6","note":"快速购买2","pointerId":3,"posX":130,"posY":370,"time":80,"type":"Tap"},{"key":"M3","note":"观察","pointerId":4,"posX":1000,"posY":200,"scale":1,"type":"Observation"},{"key":"Tab","macro":{"down":[{"args":["default",5,1185,40,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1220,100,80],"type":"touch"}]},"note":"战绩面板","posX":1185,"posY":40,"type":"Macro"},{"key":"ShiftLeft","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1200,60,80],"type":"touch"}]},"note":"商店","posX":44,"posY":302,"type":"Macro"},{"key":"KeyZ","macro":{"down":[{"args":["default",5,250,230,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,640,150,80],"type":"touch"}]},"note":"地图","posX":250,"posY":230,"type":"Macro"},{"key":"Backquote","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,510,630,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1165,575,80],"type":"touch"},{"args":["default",5,1200,65,80],"type":"touch"},{"args":[200],"type":"sleep"},{"args":["default",5,125,300,80],"type":"touch"}],"loop":null,"up":null},"note":"换装2","posX":236,"posY":66,"type":"Macro"},{"key":"WheelDown","note":"","pointerId":2,"posX":1189,"posY":422,"range":50,"type":"DirectionalSkill"},{"directional":false,"key":"M4","note":"","pointerId":2,"posX":951,"posY":636,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"type":"Macro","key":"Enter","note":"聊天","posX":1250,"posY":307,"macro":{"down":[{"args":["default",5,1245,280,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1160,560,80],"type":"touch"},{"args":[1],"type":"input-text"}],"loop":null,"up":null}},{"type":"TriggerWhenPressedSkill","key":"WheelUp","note":"装备技能","posX":1157,"posY":276,"pointerId":2,"directional":false,"rangeOrTime":80}],"relativeSize":{"h":720,"w":1280},"title":"AVD-王者荣耀-四技能-导入默认"},
+    {"list":[{"key":{"down":"KeyS","left":"KeyA","right":"KeyD","up":"KeyW"},"note":"方向轮盘","offset":175,"pointerId":1,"posX":183,"posY":566,"type":"SteeringWheel"},{"key":"KeyQ","note":"技能1","pointerId":2,"posX":952,"posY":636,"range":50,"type":"DirectionalSkill"},{"key":"AltLeft","note":"技能2","pointerId":2,"posX":979,"posY":526,"range":50,"type":"DirectionalSkill"},{"key":"KeyE","note":"技能3","pointerId":2,"posX":1074,"posY":438,"range":50,"type":"DirectionalSkill"},{"key":"Space","note":"取消技能","pointerId":2,"posX":1160,"posY":140,"type":"CancelSkill"},{"key":"KeyB","note":"回城","pointerId":3,"posX":644,"posY":650,"time":80,"type":"Tap"},{"key":"KeyC","note":"回复","pointerId":3,"posX":742,"posY":650,"time":80,"type":"Tap"},{"key":"KeyF","note":"召唤师技能","pointerId":2,"posX":838,"posY":647,"range":50,"type":"DirectionalSkill"},{"key":"M2","note":"攻击","pointerId":3,"posX":1179,"posY":621,"time":80,"type":"Tap"},{"key":"Digit1","note":"技能1升级","pointerId":3,"posX":895,"posY":564,"time":80,"type":"Tap"},{"key":"Digit2","note":"技能2升级","pointerId":3,"posX":922,"posY":456,"time":80,"type":"Tap"},{"key":"Digit3","note":"技能3升级","pointerId":3,"posX":1015,"posY":376,"time":80,"type":"Tap"},{"key":"Digit5","note":"快速购买1","pointerId":3,"posX":133,"posY":289,"time":80,"type":"Tap"},{"key":"Digit6","note":"快速购买2","pointerId":3,"posX":130,"posY":370,"time":80,"type":"Tap"},{"key":"M3","note":"观察","pointerId":4,"posX":1000,"posY":200,"scale":1,"type":"Observation"},{"key":"Tab","macro":{"down":[{"args":["default",5,1185,40,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1220,100,80],"type":"touch"}]},"note":"战绩面板","posX":1185,"posY":40,"type":"Macro"},{"key":"ShiftLeft","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1200,60,80],"type":"touch"}]},"note":"商店","posX":44,"posY":302,"type":"Macro"},{"key":"KeyZ","macro":{"down":[{"args":["default",5,250,230,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,640,150,80],"type":"touch"}]},"note":"地图","posX":250,"posY":230,"type":"Macro"},{"key":"Backquote","macro":{"down":[{"args":["default",5,40,300,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,510,630,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1165,575,80],"type":"touch"},{"args":["default",5,1200,65,80],"type":"touch"},{"args":[200],"type":"sleep"},{"args":["default",5,125,300,80],"type":"touch"}],"loop":null,"up":null},"note":"换装2","posX":236,"posY":66,"type":"Macro"},{"key":"WheelDown","note":"","pointerId":2,"posX":1189,"posY":422,"range":50,"type":"DirectionalSkill"},{"directional":false,"key":"M4","note":"","pointerId":2,"posX":951,"posY":636,"rangeOrTime":80,"type":"TriggerWhenPressedSkill"},{"type":"Macro","key":"Enter","note":"聊天","posX":1250,"posY":307,"macro":{"down":[{"args":["default",5,1245,280,80],"type":"touch"},{"args":[150],"type":"sleep"},{"args":["default",5,1160,560,80],"type":"touch"},{"args":[],"type":"key-input-mode"}],"loop":null,"up":null}},{"type":"TriggerWhenPressedSkill","key":"WheelUp","note":"装备技能","posX":1157,"posY":276,"pointerId":2,"directional":false,"rangeOrTime":80}],"relativeSize":{"h":720,"w":1280},"title":"AVD-王者荣耀-四技能-导入默认"},
     {"list":[{"key":"Backquote","note":"准星键","pointerId":0,"posX":640,"posY":361,"scaleX":1,"scaleY":1,"type":"Sight"},{"key":{"down":"KeyS","left":"KeyA","right":"KeyD","up":"KeyW"},"note":"","offset":150,"pointerId":1,"posX":208,"posY":542,"type":"SteeringWheel"},{"key":"KeyC","note":"","pointerId":3,"posX":1085,"posY":668,"time":80,"type":"Tap"},{"key":"KeyZ","note":"","pointerId":3,"posX":1207,"posY":648,"time":80,"type":"Tap"},{"key":"Space","note":"","pointerId":3,"posX":1230,"posY":503,"time":80,"type":"Tap"},{"key":"M3","note":"","pointerId":4,"posX":992,"posY":229,"scale":0.6,"type":"Observation"},{"key":"Digit1","note":"","pointerId":3,"posX":543,"posY":642,"time":80,"type":"Tap"},{"key":"Digit2","note":"","pointerId":3,"posX":729,"posY":644,"time":80,"type":"Tap"},{"key":"ShiftLeft","note":"","pointerId":3,"posX":1075,"posY":194,"time":80,"type":"Tap"},{"key":"KeyF","note":"","pointerId":3,"posX":861,"posY":310,"time":80,"type":"Tap"},{"key":"KeyG","note":"","pointerId":3,"posX":865,"posY":388,"time":80,"type":"Tap"},{"key":"KeyR","note":"","pointerId":3,"posX":976,"posY":674,"time":80,"type":"Tap"},{"key":"KeyE","note":"","pointerId":3,"posX":872,"posY":495,"time":80,"type":"Tap"},{"key":"KeyB","note":"","pointerId":3,"posX":543,"posY":593,"time":80,"type":"Tap"},{"key":"KeyN","note":"","pointerId":3,"posX":721,"posY":589,"time":80,"type":"Tap"},{"drag":false,"note":"","pointerId":2,"posX":1097,"posY":549,"scaleX":0.5,"scaleY":0.5,"type":"Fire"},{"key":"M2","note":"","pointerId":3,"posX":1227,"posY":376,"time":80,"type":"Tap"},{"key":"Tab","macro":{"down":[{"args":["default",5,100,650,80],"type":"touch"}],"loop":null,"up":[{"args":["default",5,1250,40,80],"type":"touch"}]},"note":"背包宏","posX":95,"posY":656,"type":"Macro"},{"key":"M1","note":"地图","pointerId":3,"posX":1244,"posY":43,"time":80,"type":"Tap"},{"key":"KeyV","note":"","pointerId":3,"posX":1156,"posY":300,"time":80,"type":"Tap"},{"key":"M4","note":"","pointerId":3,"posX":862,"posY":180,"time":80,"type":"Tap"},{"key":"KeyT","note":"","pointerId":3,"posX":880,"posY":674,"time":80,"type":"Tap"},{"key":"KeyQ","note":"","pointerId":3,"posX":400,"posY":679,"time":80,"type":"Tap"},{"key":"Digit4","note":"","pointerId":3,"posX":766,"posY":251,"time":80,"type":"Tap"},{"key":"Digit5","note":"","pointerId":3,"posX":770,"posY":320,"time":80,"type":"Tap"},{"key":"Digit3","note":"","pointerId":3,"posX":788,"posY":591,"time":80,"type":"Tap"}],"relativeSize":{"h":720,"w":1280},"title":"AVD-和平精英-导入默认"}
 ]

src-tauri/src/adb.rs

@@ -88,7 +88,6 @@ pub struct Adb;
 impl Adb {
     pub fn cmd_base() -> Command {
         let adb_path = share::ADB_PATH.lock().unwrap().clone();
-        println!("{}", &adb_path);
         #[cfg(target_os = "windows")]
         {
             let mut cmd = Command::new(adb_path);

src-tauri/src/main.rs

@@ -91,8 +91,8 @@ fn start_scrcpy_server(
             let sender = front_msg_sender.clone();
             // println!("收到front-command: {}", event.payload());
             tokio::spawn(async move {
-                if let Err(e) = sender.send(event.payload().into()).await {
+                if let Err(_) = sender.send(event.payload().into()).await {
-                    println!("front-command转发失败: {}", e);
+                    println!("front-command forwarding failure, please restart the program !");
                 };
             });
         });

src-tauri/tauri.conf.json

@@ -1,6 +1,6 @@
 {
   "productName": "scrcpy-mask",
-  "version": "0.4.2",
+  "version": "0.4.3",
   "identifier": "com.akichase.mask",
   "build": {
     "beforeDevCommand": "pnpm dev",

src/components/Device.vue

@@ -284,7 +284,8 @@ async function deviceGetScreenSize() {
   const size = await getDeviceScreenSize(id);
   store.hideLoading();
   message.success(
-    t("pages.Device.deviceGetScreenSize") + `${size[0]} x ${size[1]}`
+    t("pages.Device.deviceGetScreenSize") + `${size[0]} x ${size[1]}`,
+    { keepAliveOnHover: true }
   );
 }
 

src/components/Mask.vue

@@ -1,15 +1,10 @@
 <script setup lang="ts">
-import { h, nextTick, onActivated, onMounted, ref } from "vue";
+import { h, onActivated, onMounted } from "vue";
-import {
+import { MessageReactive, NDialog, useDialog, useMessage } from "naive-ui";
-  MessageReactive,
-  NDialog,
-  NInput,
-  useDialog,
-  useMessage,
-} from "naive-ui";
 import { useGlobalStore } from "../store/global";
 import { onBeforeRouteLeave, useRouter } from "vue-router";
 import {
+  KeyInputHandler,
   applyShortcuts,
   clearShortcuts,
   listenToEvent,
@@ -19,12 +14,9 @@ import { KeyMappingConfig, KeySteeringWheel } from "../keyMappingConfig";
 import { getVersion } from "@tauri-apps/api/app";
 import { fetch } from "@tauri-apps/plugin-http";
 import { open } from "@tauri-apps/plugin-shell";
-import { sendSetClipboard } from "../frontcommand/controlMsg";
 import { getCurrent, PhysicalSize } from "@tauri-apps/api/window";
-import { AndroidKeycode } from "../frontcommand/android";
 import { Store } from "@tauri-apps/plugin-store";
 import { useI18n } from "vue-i18n";
-import { SendKeyAction, sendKey } from "../frontcommand/scrcpyMaskCmd";
 import { checkAdbAvailable } from "../invoke";
 
 const { t } = useI18n();
@@ -33,14 +25,11 @@ const router = useRouter();
 const message = useMessage();
 const dialog = useDialog();
 
-const showInputBoxRef = ref(false);
-const inputBoxVal = ref("");
-const inputInstRef = ref<HTMLInputElement | null>(null);
-
 onBeforeRouteLeave(() => {
   if (store.controledDevice) {
     unlistenToEvent();
     clearShortcuts();
+    if (store.keyInputFlag) KeyInputHandler.removeEventListener();
   }
 });
 
@@ -68,7 +57,6 @@ onActivated(async () => {
 onMounted(async () => {
   await loadLocalStore();
   store.checkUpdate = checkUpdate;
-  store.showInputBox = showInputBox;
   if (store.checkUpdateAtStart) checkUpdate();
   store.checkAdb = checkAdb;
   setTimeout(() => {
@@ -159,64 +147,6 @@ async function cleanAfterimage() {
   await appWindow.setSize(oldSize);
 }
 
-function handleInputBoxClick(event: MouseEvent) {
-  if (event.target === document.getElementById("input-box")) {
-    showInputBox(false);
-  }
-}
-
-function handleInputKeyUp(event: KeyboardEvent) {
-  if (event.key === "Enter") {
-    pasteText();
-  } else if (event.key === "Escape") {
-    showInputBox(false);
-  }
-}
-
-function showInputBox(flag: boolean) {
-  if (flag) {
-    unlistenToEvent();
-    inputBoxVal.value = "";
-    showInputBoxRef.value = true;
-    document.addEventListener("keyup", handleInputKeyUp);
-    nextTick(() => {
-      inputInstRef.value?.focus();
-    });
-  } else {
-    document.removeEventListener("keyup", handleInputKeyUp);
-    inputInstRef.value?.blur();
-    showInputBoxRef.value = false;
-    listenToEvent();
-    nextTick(() => {
-      cleanAfterimage();
-    });
-  }
-}
-
-function sleep(time: number) {
-  return new Promise<void>((resolve) => {
-    setTimeout(() => {
-      resolve();
-    }, time);
-  });
-}
-
-async function pasteText() {
-  showInputBox(false);
-  if (!inputBoxVal.value) return;
-  sendSetClipboard({
-    sequence: new Date().getTime() % 100000,
-    text: inputBoxVal.value,
-    paste: true,
-  });
-  await sleep(300);
-  // send enter
-  await sendKey({
-    action: SendKeyAction.Default,
-    keycode: AndroidKeycode.AKEYCODE_ENTER,
-  });
-}
-
 function toStartServer() {
   router.replace({ name: "device" });
 }
@@ -296,19 +226,6 @@ async function checkUpdate() {
   </div>
   <template v-if="store.keyMappingConfigList.length">
     <div @contextmenu.prevent class="mask" id="maskElement"></div>
-    <div
-      v-show="showInputBoxRef"
-      class="input-box"
-      id="input-box"
-      @click="handleInputBoxClick"
-    >
-      <NInput
-        ref="inputInstRef"
-        v-model:value="inputBoxVal"
-        type="text"
-        :placeholder="$t('pages.Mask.inputBoxPlaceholder')"
-      />
-    </div>
     <div
       v-if="store.maskButton.show"
       :style="'--transparency: ' + store.maskButton.transparency"
@@ -369,26 +286,6 @@ async function checkUpdate() {
   z-index: 2;
 }
 
-.input-box {
-  z-index: 4;
-  position: absolute;
-  left: 70px;
-  top: 30px;
-  right: 0;
-  bottom: 0;
-  background-color: rgba(0, 0, 0, 0.5);
-
-  .n-input {
-    width: 50%;
-    position: absolute;
-    left: 0;
-    right: 0;
-    bottom: 15%;
-    margin: auto;
-    background-color: var(--content-bg-color);
-  }
-}
-
 .button-layer {
   position: absolute;
   left: 70px;

src/frontcommand/KeyToCodeMap.ts

@@ -0,0 +1,118 @@
+import { UIEventsCode } from "./UIEventsCode";
+import { AndroidKeycode } from "./android";
+
+export const KeyToCodeMap = new Map([
+  [UIEventsCode.Backquote, AndroidKeycode.AKEYCODE_GRAVE],
+  [UIEventsCode.Backslash, AndroidKeycode.AKEYCODE_BACKSLASH],
+  [UIEventsCode.BracketLeft, AndroidKeycode.AKEYCODE_LEFT_BRACKET],
+  [UIEventsCode.BracketRight, AndroidKeycode.AKEYCODE_RIGHT_BRACKET],
+  [UIEventsCode.Comma, AndroidKeycode.AKEYCODE_COMMA],
+  [UIEventsCode.Digit0, AndroidKeycode.AKEYCODE_0],
+  [UIEventsCode.Digit1, AndroidKeycode.AKEYCODE_1],
+  [UIEventsCode.Digit2, AndroidKeycode.AKEYCODE_2],
+  [UIEventsCode.Digit3, AndroidKeycode.AKEYCODE_3],
+  [UIEventsCode.Digit4, AndroidKeycode.AKEYCODE_4],
+  [UIEventsCode.Digit5, AndroidKeycode.AKEYCODE_5],
+  [UIEventsCode.Digit6, AndroidKeycode.AKEYCODE_6],
+  [UIEventsCode.Digit7, AndroidKeycode.AKEYCODE_7],
+  [UIEventsCode.Digit8, AndroidKeycode.AKEYCODE_8],
+  [UIEventsCode.Digit9, AndroidKeycode.AKEYCODE_9],
+  [UIEventsCode.Equal, AndroidKeycode.AKEYCODE_EQUALS],
+  [UIEventsCode.IntlRo, AndroidKeycode.AKEYCODE_RO],
+  [UIEventsCode.IntlYen, AndroidKeycode.AKEYCODE_YEN],
+  [UIEventsCode.KeyA, AndroidKeycode.AKEYCODE_A],
+  [UIEventsCode.KeyB, AndroidKeycode.AKEYCODE_B],
+  [UIEventsCode.KeyC, AndroidKeycode.AKEYCODE_C],
+  [UIEventsCode.KeyD, AndroidKeycode.AKEYCODE_D],
+  [UIEventsCode.KeyE, AndroidKeycode.AKEYCODE_E],
+  [UIEventsCode.KeyF, AndroidKeycode.AKEYCODE_F],
+  [UIEventsCode.KeyG, AndroidKeycode.AKEYCODE_G],
+  [UIEventsCode.KeyH, AndroidKeycode.AKEYCODE_H],
+  [UIEventsCode.KeyI, AndroidKeycode.AKEYCODE_I],
+  [UIEventsCode.KeyJ, AndroidKeycode.AKEYCODE_J],
+  [UIEventsCode.KeyK, AndroidKeycode.AKEYCODE_K],
+  [UIEventsCode.KeyL, AndroidKeycode.AKEYCODE_L],
+  [UIEventsCode.KeyM, AndroidKeycode.AKEYCODE_M],
+  [UIEventsCode.KeyN, AndroidKeycode.AKEYCODE_N],
+  [UIEventsCode.KeyO, AndroidKeycode.AKEYCODE_O],
+  [UIEventsCode.KeyP, AndroidKeycode.AKEYCODE_P],
+  [UIEventsCode.KeyQ, AndroidKeycode.AKEYCODE_Q],
+  [UIEventsCode.KeyR, AndroidKeycode.AKEYCODE_R],
+  [UIEventsCode.KeyS, AndroidKeycode.AKEYCODE_S],
+  [UIEventsCode.KeyT, AndroidKeycode.AKEYCODE_T],
+  [UIEventsCode.KeyU, AndroidKeycode.AKEYCODE_U],
+  [UIEventsCode.KeyV, AndroidKeycode.AKEYCODE_V],
+  [UIEventsCode.KeyW, AndroidKeycode.AKEYCODE_W],
+  [UIEventsCode.KeyX, AndroidKeycode.AKEYCODE_X],
+  [UIEventsCode.KeyY, AndroidKeycode.AKEYCODE_Y],
+  [UIEventsCode.KeyZ, AndroidKeycode.AKEYCODE_Z],
+  [UIEventsCode.Minus, AndroidKeycode.AKEYCODE_MINUS],
+  [UIEventsCode.Period, AndroidKeycode.AKEYCODE_PERIOD],
+  [UIEventsCode.Quote, AndroidKeycode.AKEYCODE_APOSTROPHE],
+  [UIEventsCode.Semicolon, AndroidKeycode.AKEYCODE_SEMICOLON],
+  [UIEventsCode.Slash, AndroidKeycode.AKEYCODE_SLASH],
+  [UIEventsCode.KanaMode, AndroidKeycode.AKEYCODE_KANA],
+  [UIEventsCode.Delete, AndroidKeycode.AKEYCODE_FORWARD_DEL],
+  [UIEventsCode.End, AndroidKeycode.AKEYCODE_MOVE_END],
+  [UIEventsCode.Help, AndroidKeycode.AKEYCODE_HELP],
+  [UIEventsCode.Home, AndroidKeycode.AKEYCODE_MOVE_HOME],
+  [UIEventsCode.Insert, AndroidKeycode.AKEYCODE_INSERT],
+  [UIEventsCode.PageDown, AndroidKeycode.AKEYCODE_PAGE_DOWN],
+  [UIEventsCode.PageUp, AndroidKeycode.AKEYCODE_PAGE_UP],
+  [UIEventsCode.AltLeft, AndroidKeycode.AKEYCODE_ALT_LEFT],
+  [UIEventsCode.AltRight, AndroidKeycode.AKEYCODE_ALT_RIGHT],
+  [UIEventsCode.Backspace, AndroidKeycode.AKEYCODE_DEL],
+  [UIEventsCode.CapsLock, AndroidKeycode.AKEYCODE_CAPS_LOCK],
+  [UIEventsCode.ControlLeft, AndroidKeycode.AKEYCODE_CTRL_LEFT],
+  [UIEventsCode.ControlRight, AndroidKeycode.AKEYCODE_CTRL_RIGHT],
+  [UIEventsCode.Enter, AndroidKeycode.AKEYCODE_ENTER],
+  [UIEventsCode.MetaLeft, AndroidKeycode.AKEYCODE_META_LEFT],
+  [UIEventsCode.MetaRight, AndroidKeycode.AKEYCODE_META_RIGHT],
+  [UIEventsCode.ShiftLeft, AndroidKeycode.AKEYCODE_SHIFT_LEFT],
+  [UIEventsCode.ShiftRight, AndroidKeycode.AKEYCODE_SHIFT_RIGHT],
+  [UIEventsCode.Space, AndroidKeycode.AKEYCODE_SPACE],
+  [UIEventsCode.Tab, AndroidKeycode.AKEYCODE_TAB],
+  [UIEventsCode.ArrowLeft, AndroidKeycode.AKEYCODE_DPAD_LEFT],
+  [UIEventsCode.ArrowUp, AndroidKeycode.AKEYCODE_DPAD_UP],
+  [UIEventsCode.ArrowRight, AndroidKeycode.AKEYCODE_DPAD_RIGHT],
+  [UIEventsCode.ArrowDown, AndroidKeycode.AKEYCODE_DPAD_DOWN],
+
+  [UIEventsCode.NumLock, AndroidKeycode.AKEYCODE_NUM_LOCK],
+  [UIEventsCode.Numpad0, AndroidKeycode.AKEYCODE_NUMPAD_0],
+  [UIEventsCode.Numpad1, AndroidKeycode.AKEYCODE_NUMPAD_1],
+  [UIEventsCode.Numpad2, AndroidKeycode.AKEYCODE_NUMPAD_2],
+  [UIEventsCode.Numpad3, AndroidKeycode.AKEYCODE_NUMPAD_3],
+  [UIEventsCode.Numpad4, AndroidKeycode.AKEYCODE_NUMPAD_4],
+  [UIEventsCode.Numpad5, AndroidKeycode.AKEYCODE_NUMPAD_5],
+  [UIEventsCode.Numpad6, AndroidKeycode.AKEYCODE_NUMPAD_6],
+  [UIEventsCode.Numpad7, AndroidKeycode.AKEYCODE_NUMPAD_7],
+  [UIEventsCode.Numpad8, AndroidKeycode.AKEYCODE_NUMPAD_8],
+  [UIEventsCode.Numpad9, AndroidKeycode.AKEYCODE_NUMPAD_9],
+  [UIEventsCode.NumpadAdd, AndroidKeycode.AKEYCODE_NUMPAD_ADD],
+  [UIEventsCode.NumpadComma, AndroidKeycode.AKEYCODE_NUMPAD_COMMA],
+  [UIEventsCode.NumpadDecimal, AndroidKeycode.AKEYCODE_NUMPAD_DOT],
+  [UIEventsCode.NumpadDivide, AndroidKeycode.AKEYCODE_NUMPAD_DIVIDE],
+  [UIEventsCode.NumpadEnter, AndroidKeycode.AKEYCODE_NUMPAD_ENTER],
+  [UIEventsCode.NumpadEqual, AndroidKeycode.AKEYCODE_NUMPAD_EQUALS],
+  [UIEventsCode.NumpadMultiply, AndroidKeycode.AKEYCODE_NUMPAD_MULTIPLY],
+  [UIEventsCode.NumpadParenLeft, AndroidKeycode.AKEYCODE_NUMPAD_LEFT_PAREN],
+  [UIEventsCode.NumpadParenRight, AndroidKeycode.AKEYCODE_NUMPAD_RIGHT_PAREN],
+  [UIEventsCode.NumpadSubtract, AndroidKeycode.AKEYCODE_NUMPAD_SUBTRACT],
+
+  [UIEventsCode.Escape, AndroidKeycode.AKEYCODE_ESCAPE],
+  [UIEventsCode.F1, AndroidKeycode.AKEYCODE_F1],
+  [UIEventsCode.F2, AndroidKeycode.AKEYCODE_F2],
+  [UIEventsCode.F3, AndroidKeycode.AKEYCODE_F3],
+  [UIEventsCode.F4, AndroidKeycode.AKEYCODE_F4],
+  [UIEventsCode.F5, AndroidKeycode.AKEYCODE_F5],
+  [UIEventsCode.F6, AndroidKeycode.AKEYCODE_F6],
+  [UIEventsCode.F7, AndroidKeycode.AKEYCODE_F7],
+  [UIEventsCode.F8, AndroidKeycode.AKEYCODE_F8],
+  [UIEventsCode.F9, AndroidKeycode.AKEYCODE_F9],
+  [UIEventsCode.F10, AndroidKeycode.AKEYCODE_F10],
+  [UIEventsCode.F11, AndroidKeycode.AKEYCODE_F11],
+  [UIEventsCode.F12, AndroidKeycode.AKEYCODE_F12],
+  [UIEventsCode.Fn, AndroidKeycode.AKEYCODE_FUNCTION],
+  [UIEventsCode.PrintScreen, AndroidKeycode.AKEYCODE_SYSRQ],
+  [UIEventsCode.Pause, AndroidKeycode.AKEYCODE_BREAK],
+]);

src/frontcommand/UIEventsCode.ts

@@ -0,0 +1,191 @@
+// https://w3c.github.io/uievents-code/
+
+export enum UIEventsCode {
+  // 3.1.1.1. Writing System Keys
+  Backquote = "Backquote",
+  Backslash = "Backslash",
+  BracketLeft = "BracketLeft",
+  BracketRight = "BracketRight",
+  Comma = "Comma",
+  Digit0 = "Digit0",
+  Digit1 = "Digit1",
+  Digit2 = "Digit2",
+  Digit3 = "Digit3",
+  Digit4 = "Digit4",
+  Digit5 = "Digit5",
+  Digit6 = "Digit6",
+  Digit7 = "Digit7",
+  Digit8 = "Digit8",
+  Digit9 = "Digit9",
+  Equal = "Equal",
+  IntlBackslash = "IntlBackslash",
+  IntlRo = "IntlRo",
+  IntlYen = "IntlYen",
+  KeyA = "KeyA",
+  KeyB = "KeyB",
+  KeyC = "KeyC",
+  KeyD = "KeyD",
+  KeyE = "KeyE",
+  KeyF = "KeyF",
+  KeyG = "KeyG",
+  KeyH = "KeyH",
+  KeyI = "KeyI",
+  KeyJ = "KeyJ",
+  KeyK = "KeyK",
+  KeyL = "KeyL",
+  KeyM = "KeyM",
+  KeyN = "KeyN",
+  KeyO = "KeyO",
+  KeyP = "KeyP",
+  KeyQ = "KeyQ",
+  KeyR = "KeyR",
+  KeyS = "KeyS",
+  KeyT = "KeyT",
+  KeyU = "KeyU",
+  KeyV = "KeyV",
+  KeyW = "KeyW",
+  KeyX = "KeyX",
+  KeyY = "KeyY",
+  KeyZ = "KeyZ",
+  Minus = "Minus",
+  Period = "Period",
+  Quote = "Quote",
+  Semicolon = "Semicolon",
+  Slash = "Slash",
+
+  // 3.1.1.2. Functional Keys
+  AltLeft = "AltLeft",
+  AltRight = "AltRight",
+  Backspace = "Backspace",
+  CapsLock = "CapsLock",
+  ContextMenu = "ContextMenu",
+  ControlLeft = "ControlLeft",
+  ControlRight = "ControlRight",
+  Enter = "Enter",
+  MetaLeft = "MetaLeft",
+  MetaRight = "MetaRight",
+  ShiftLeft = "ShiftLeft",
+  ShiftRight = "ShiftRight",
+  Space = "Space",
+  Tab = "Tab",
+  Convert = "Convert",
+  KanaMode = "KanaMode",
+  Lang1 = "Lang1",
+  Lang2 = "Lang2",
+  Lang3 = "Lang3",
+  Lang4 = "Lang4",
+  Lang5 = "Lang5",
+  NonConvert = "NonConvert",
+
+  // 3.1.2. Control Pad Section
+  Delete = "Delete",
+  End = "End",
+  Help = "Help",
+  Home = "Home",
+  Insert = "Insert",
+  PageDown = "PageDown",
+  PageUp = "PageUp",
+
+  // 3.1.3. Arrow Pad Section
+  ArrowDown = "ArrowDown",
+  ArrowLeft = "ArrowLeft",
+  ArrowRight = "ArrowRight",
+  ArrowUp = "ArrowUp",
+
+  // 3.1.4. Numpad Section
+  NumLock = "NumLock",
+  Numpad0 = "Numpad0",
+  Numpad1 = "Numpad1",
+  Numpad2 = "Numpad2",
+  Numpad3 = "Numpad3",
+  Numpad4 = "Numpad4",
+  Numpad5 = "Numpad5",
+  Numpad6 = "Numpad6",
+  Numpad7 = "Numpad7",
+  Numpad8 = "Numpad8",
+  Numpad9 = "Numpad9",
+  NumpadAdd = "NumpadAdd",
+  NumpadBackspace = "NumpadBackspace",
+  NumpadClear = "NumpadClear",
+  NumpadClearEntry = "NumpadClearEntry",
+  NumpadComma = "NumpadComma",
+  NumpadDecimal = "NumpadDecimal",
+  NumpadDivide = "NumpadDivide",
+  NumpadEnter = "NumpadEnter",
+  NumpadEqual = "NumpadEqual",
+  NumpadHash = "NumpadHash",
+  NumpadMemoryAdd = "NumpadMemoryAdd",
+  NumpadMemoryClear = "NumpadMemoryClear",
+  NumpadMemoryRecall = "NumpadMemoryRecall",
+  NumpadMemoryStore = "NumpadMemoryStore",
+  NumpadMemorySubtract = "NumpadMemorySubtract",
+  NumpadMultiply = "NumpadMultiply",
+  NumpadParenLeft = "NumpadParenLeft",
+  NumpadParenRight = "NumpadParenRight",
+  NumpadStar = "NumpadStar",
+  NumpadSubtract = "NumpadSubtract",
+
+  // 3.1.5. Function Section
+  Escape = "Escape",
+  F1 = "F1",
+  F2 = "F2",
+  F3 = "F3",
+  F4 = "F4",
+  F5 = "F5",
+  F6 = "F6",
+  F7 = "F7",
+  F8 = "F8",
+  F9 = "F9",
+  F10 = "F10",
+  F11 = "F11",
+  F12 = "F12",
+  Fn = "Fn",
+  FnLock = "FnLock",
+  PrintScreen = "PrintScreen",
+  ScrollLock = "ScrollLock",
+  Pause = "Pause",
+
+  // 3.1.6. Media Keys
+  BrowserBack = "BrowserBack",
+  BrowserFavorites = "BrowserFavorites",
+  BrowserForward = "BrowserForward",
+  BrowserHome = "BrowserHome",
+  BrowserRefresh = "BrowserRefresh",
+  BrowserSearch = "BrowserSearch",
+  BrowserStop = "BrowserStop",
+  Eject = "Eject",
+  LaunchApp1 = "LaunchApp1",
+  LaunchApp2 = "LaunchApp2",
+  LaunchMail = "LaunchMail",
+  MediaPlayPause = "MediaPlayPause",
+  MediaSelect = "MediaSelect",
+  MediaStop = "MediaStop",
+  MediaTrackNext = "MediaTrackNext",
+  MediaTrackPrevious = "MediaTrackPrevious",
+  Power = "Power",
+  Sleep = "Sleep",
+  AudioVolumeDown = "AudioVolumeDown",
+  AudioVolumeMute = "AudioVolumeMute",
+  AudioVolumeUp = "AudioVolumeUp",
+  WakeUp = "WakeUp",
+
+  // 3.1.7. Legacy, Non-Standard and Special Keys
+  Hyper = "Hyper",
+  Super = "Super",
+  Turbo = "Turbo",
+  Abort = "Abort",
+  Resume = "Resume",
+  Suspend = "Suspend",
+  Again = "Again",
+  Copy = "Copy",
+  Cut = "Cut",
+  Find = "Find",
+  Open = "Open",
+  Paste = "Paste",
+  Props = "Props",
+  Select = "Select",
+  Undo = "Undo",
+  Hiragana = "Hiragana",
+  Katakana = "Katakana",
+  Unidentified = "Unidentified",
+}

src/hotkey.ts

@@ -1,5 +1,5 @@
 // https://github.com/jamiebuilds/tinykeys/pull/193/commits/2598ecb3db6b3948c7acbf0e7bd8b0674961ad61
-import { useMessage } from "naive-ui";
+import { MessageReactive, useMessage } from "naive-ui";
 import {
   SwipeAction,
   TouchAction,
@@ -25,6 +25,13 @@ import { useGlobalStore } from "./store/global";
 import { LogicalPosition, getCurrent } from "@tauri-apps/api/window";
 import { useI18n } from "vue-i18n";
 import { UnlistenFn } from "@tauri-apps/api/event";
+import { KeyToCodeMap } from "./frontcommand/KeyToCodeMap";
+import {
+  AndroidKeyEventAction,
+  AndroidMetastate,
+} from "./frontcommand/android";
+import { UIEventsCode } from "./frontcommand/UIEventsCode";
+import { sendInjectKeycode } from "./frontcommand/controlMsg";
 
 function clientxToPosx(clientx: number) {
   return clientx < 70
@@ -646,6 +653,7 @@ function addClickShortcuts(key: string, pointerId: number) {
   );
 }
 
+// add shortcut for sight mode
 function addSightShortcuts(
   relativeSize: { w: number; h: number },
   sightKeyMapping: KeySight,
@@ -793,12 +801,23 @@ function addSightShortcuts(
   addShortcut(sightKeyMapping.key, async () => {
     if (mouseLock) {
       // stop sight mode
+
+      // remove box element
+      const mouseRangeBoxElement = document.getElementById("mouseRangeBox");
+      if (mouseRangeBoxElement) {
+        mouseRangeBoxElement.removeEventListener(
+          "mouseleave",
+          moveLeaveHandler
+        );
+        mouseRangeBoxElement.remove();
+      }
+      maskElement.style.cursor = "pointer";
+
+      mouseLock = false;
       loopDownKeyCBMap.delete(sightKeyMapping.key);
       await touchRelateToSight(TouchAction.Up);
       await appWindow.setCursorVisible(true);
-      maskElement.removeEventListener("mouseleave", moveLeaveHandler);
+
-      maskElement.style.cursor = "pointer";
-      mouseLock = false;
       if (msgReactive) {
         msgReactive.destroy();
         msgReactive = null;
@@ -811,10 +830,15 @@ function addSightShortcuts(
       addClickShortcuts("M0", 0);
     } else {
       // start sight mode
-      await appWindow.setCursorVisible(false);
+
-      maskElement.addEventListener("mouseleave", moveLeaveHandler);
+      // create box element
       maskElement.style.cursor = "none";
+      const mouseRangeBoxElement = createMouseRangeBox();
+      mouseRangeBoxElement.addEventListener("mouseleave", moveLeaveHandler);
+      document.body.appendChild(mouseRangeBoxElement);
+
       mouseLock = true;
+      await appWindow.setCursorVisible(false);
       msgReactive = message.info(
         t("pages.Mask.sightMode", [sightKeyMapping.key]),
         {
@@ -918,6 +942,20 @@ function addSightShortcuts(
   });
 }
 
+function createMouseRangeBox(): HTMLElement {
+  const box = document.createElement("div");
+  box.id = "mouseRangeBox";
+  box.style.position = "absolute";
+  box.style.top = "40px";
+  box.style.bottom = "40px";
+  box.style.left = "100px";
+  box.style.right = "100px";
+  box.style.zIndex = "9999";
+  box.style.backgroundColor = "transparent";
+  box.style.cursor = "none";
+  return box;
+}
+
 function handleKeydown(event: KeyboardEvent) {
   event.preventDefault();
   if (event.repeat) return;
@@ -940,7 +978,10 @@ function handleKeyup(event: KeyboardEvent) {
 }
 
 function handleMouseDown(event: MouseEvent) {
-  if (event.target !== maskElement) return;
+  const target = event.target;
+  if (!(target instanceof HTMLElement)) return;
+  if (target.id !== "maskElement" && target.id !== "mouseRangeBox") return;
+
   mouseX = event.clientX;
   mouseY = event.clientY;
   event.preventDefault();
@@ -989,6 +1030,86 @@ function handleMouseWheel(event: WheelEvent) {
   }
 }
 
+export class KeyInputHandler {
+  private static readonly repeatCounter: Map<number, number> = new Map();
+  private static msgReactive: MessageReactive | null = null;
+  private static handler(event: KeyboardEvent) {
+    const keycode = KeyToCodeMap.get(event.code as UIEventsCode);
+    if (!keycode) {
+      return;
+    }
+    let action: AndroidKeyEventAction;
+    let repeatCount = 0;
+    if (event.type === "keydown") {
+      action = AndroidKeyEventAction.AKEY_EVENT_ACTION_DOWN;
+      if (event.repeat) {
+        let count = KeyInputHandler.repeatCounter.get(keycode);
+        if (typeof count !== "number") {
+          count = 1;
+        } else {
+          count++;
+        }
+        repeatCount = count;
+        KeyInputHandler.repeatCounter.set(keycode, count);
+      }
+    } else if (event.type === "keyup") {
+      action = AndroidKeyEventAction.AKEY_EVENT_ACTION_UP;
+      KeyInputHandler.repeatCounter.delete(keycode);
+    } else {
+      return;
+    }
+    const metaState =
+      (event.getModifierState("Alt") ? AndroidMetastate.AMETA_ALT_ON : 0) |
+      (event.getModifierState("Shift") ? AndroidMetastate.AMETA_SHIFT_ON : 0) |
+      (event.getModifierState("Control") ? AndroidMetastate.AMETA_CTRL_ON : 0) |
+      (event.getModifierState("Meta") ? AndroidMetastate.AMETA_META_ON : 0) |
+      (event.getModifierState("CapsLock")
+        ? AndroidMetastate.AMETA_CAPS_LOCK_ON
+        : 0) |
+      (event.getModifierState("ScrollLock")
+        ? AndroidMetastate.AMETA_SCROLL_LOCK_ON
+        : 0) |
+      (event.getModifierState("NumLock")
+        ? AndroidMetastate.AMETA_NUM_LOCK_ON
+        : 0);
+
+    // const controlMessage = new KeyCodeControlMessage(
+    //   action,
+    //   keyCode,
+    //   repeatCount,
+    //   metaState
+    // );
+    sendInjectKeycode({
+      action,
+      keycode,
+      repeat: repeatCount,
+      metastate: metaState,
+    });
+    event.preventDefault();
+  }
+  public static addEventListener() {
+    KeyInputHandler.msgReactive = message.info(t("pages.Mask.keyInputMode"), {
+      duration: 0,
+      closable: true,
+      onClose: () => {
+        KeyInputHandler.removeEventListener();
+        listenToEvent(true);
+        store.keyInputFlag = false;
+      },
+    });
+    window.addEventListener("keydown", KeyInputHandler.handler);
+    window.addEventListener("keyup", KeyInputHandler.handler);
+  }
+  public static removeEventListener() {
+    if (KeyInputHandler.msgReactive) {
+      KeyInputHandler.msgReactive.destroy();
+      KeyInputHandler.msgReactive = null;
+    }
+    window.removeEventListener("keydown", KeyInputHandler.handler);
+    window.removeEventListener("keyup", KeyInputHandler.handler);
+  }
+}
+
 function addShortcut(
   key: string,
   downCB?: () => Promise<void>,
@@ -1047,23 +1168,23 @@ function addShortcut(
  *   {
  *    type: "touch",
  *    // op, pointerId, posX, posY
- *    args: ["down", 5, ["mouse", -10], 600],
+ *    args: ["down", 5, ["mouse", -10], 600]
  *  },
  *  // sleep 1000ms
  *  {
  *   type: "sleep",
  *  // time(ms)
- *   args: [1000],
+ *   args: [1000]
  *  },
  *  // touch up
  *  {
  *  type: "touch",
- *  args: ["up", 5, ["mouse", 10], 600],
+ *  args: ["up", 5, ["mouse", 10], 600]
  *  },
  *  // touch 1000ms
  *  {
  *  type: "touch",
- *  args: ["default", 5, ["mouse", 10], 600, 1000],
+ *  args: ["default", 5, ["mouse", 10], 600, 1000]
  *  },
  *  // swipe
  *  {
@@ -1074,18 +1195,17 @@ function addShortcut(
  *     [
  *       [
  *         ["mouse", 100],
- *         ["mouse", -100],
+ *         ["mouse", -100]
  *       ],
- *       ["mouse", "mouse"],
+ *       ["mouse", "mouse"]
  *     ],
- *     1000,
+ *     1000
- *   ],
+ *   ]
  *  },
- *  // input-text
+ *  // key-input-mode
  *  {
- *    type: "input-text",
+ *    type: "key-input-mode",
- *    // 1:on, 2:off
+ *    args: []
- *    args: [1]
  *  }
  * ]);
  */
@@ -1158,13 +1278,17 @@ async function execMacro(
             intervalBetweenPos: cmd.args[3],
           });
           break;
-        case "input-text":
+        case "key-input-mode":
-          if (cmd.args[0] === 1) {
+          if (!store.keyInputFlag) {
             // on
-            useGlobalStore().showInputBox(true);
+            unlistenToEvent(true);
+            KeyInputHandler.addEventListener();
+            store.keyInputFlag = true;
           } else {
             // off
-            useGlobalStore().showInputBox(false);
+            KeyInputHandler.removeEventListener();
+            listenToEvent(true);
+            store.keyInputFlag = false;
           }
           break;
         default:
@@ -1352,9 +1476,11 @@ async function touchX(
   });
 }
 
-export function listenToEvent() {
+export function listenToEvent(onlyKeyEvent = false) {
   window.addEventListener("keydown", handleKeydown);
   window.addEventListener("keyup", handleKeyup);
+  if (onlyKeyEvent) return;
+
   window.addEventListener("mousedown", handleMouseDown);
   window.addEventListener("mousemove", handleMouseMove);
   window.addEventListener("mouseup", handleMouseUp);
@@ -1363,9 +1489,11 @@ export function listenToEvent() {
   execLoopCB();
 }
 
-export function unlistenToEvent() {
+export function unlistenToEvent(onlyKeyEvent = false) {
   window.removeEventListener("keydown", handleKeydown);
   window.removeEventListener("keyup", handleKeyup);
+  if (onlyKeyEvent) return;
+
   window.removeEventListener("mousedown", handleMouseDown);
   window.removeEventListener("mousemove", handleMouseMove);
   window.removeEventListener("mouseup", handleMouseUp);

src/i18n/en-US.json

@@ -58,7 +58,8 @@
         "positiveText": "To control"
       },
       "sightMode": "Mouse is locked, press {0} to unlock",
-      "checkAdb": "adb is not available and the software cannot run normally. Please ensure that adb is installed on the system and added to the Path environment variable correctly: {0}"
+      "checkAdb": "adb is not available and the software cannot run normally. Please ensure that adb is installed on the system and added to the Path environment variable correctly: {0}",
+      "keyInputMode": "Has entered the keystroke input mode, close this message to exit."
     },
     "Setting": {
       "tabs": {

src/i18n/zh-CN.json

@@ -58,7 +58,8 @@
       },
       "inputBoxPlaceholder": "输入文本后按Enter/Esc",
       "sightMode": "鼠标已锁定, 按 {0} 键解锁",
-      "checkAdb": "adb不可用，软件无法正常运行，请确保系统已安装adb，并正确添加到了Path环境变量中: {0}"
+      "checkAdb": "adb不可用，软件无法正常运行，请确保系统已安装adb，并正确添加到了Path环境变量中: {0}",
+      "keyInputMode": "已进入按键输入模式，关闭本消息可退出"
     },
     "Setting": {
       "tabs": {

src/keyMappingConfig.ts

@@ -66,7 +66,7 @@ export interface KeyTap extends KeyBase {
 }
 
 export type KeyMacroList = Array<{
-  type: "touch" | "sleep" | "swipe" | "input-text";
+  type: "touch" | "sleep" | "swipe" | "key-input-mode";
   args: any[];
 }> | null;
 

src/store/global.ts

@@ -27,8 +27,6 @@ export const useGlobalStore = defineStore("global", () => {
   const controledDevice: Ref<ControledDevice | null> = ref(null);
   const editKeyMappingList: Ref<KeyMapping[]> = ref([]);
 
-  const showInputBox: (_: boolean) => void = (_: boolean) => {};
-
   let checkUpdate: () => Promise<void> = async () => {};
   let checkAdb: () => Promise<void> = async () => {};
 
@@ -75,6 +73,8 @@ export const useGlobalStore = defineStore("global", () => {
   const screenSizeW: Ref<number> = ref(0);
   const screenSizeH: Ref<number> = ref(0);
 
+  const keyInputFlag = ref(false);
+
   // persistent storage
   const keyMappingConfigList: Ref<KeyMappingConfig[]> = ref([]);
   const curKeyMappingIndex = ref(0);
@@ -94,12 +94,12 @@ export const useGlobalStore = defineStore("global", () => {
     // in-memory storage
     screenSizeW,
     screenSizeH,
+    keyInputFlag,
     showLoading,
     hideLoading,
     showLoadingRef,
     controledDevice,
     editKeyMappingList,
-    showInputBox,
     applyEditKeyMappingList,
     resetEditKeyMappingList,
     setKeyMappingIndex,