To achieve computer control of Android devices, I developed a cross-platform desktop client using Tarui + Vue 3 + Rust. This client provides visual mouse and keyboard mapping configuration, enabling multi-touch operations similar to Android emulators through key mapping, with millisecond-level response time. This tool can be widely used for controlling Android devices from computers to play mobile games, providing a smooth touch experience.
I express my deep respect and gratitude to the developers of the Scrcpy project. Scrcpy is a powerful and efficient open-source tool that greatly facilitates control over Android devices. This project is built upon the excellent architecture of Scrcpy, with optimizations and adjustments for mouse and keyboard control.
**This project does not provide screen mirroring functionality—let me emphasize, it does not provide screen mirroring functionality!** It only implements the Scrcpy control protocol.
Because screen mirroring may involve latency and blurriness issues, this project takes a different approach by directly abandoning screen mirroring and instead using a transparent mask to display the content behind the window (which can be AVD, low-latency screen mirroring provided by your phone manufacturers, etc.), Completely eliminates the problem of poor screen casting experience inherent in Scrcpy.
Furthermore, to better support interaction between Scrcpy Mask and Android devices, I have made some modifications to the scrcpy-server, leading to the creation of a separate branch project called [scrcpy-mask-server](https://github.com/AkiChase/scrcpy-mask-server).
1. You need to solve the problem of screen casting on your own. Recommend using the official screen mirror method of your device brand to achieve the minimum delay
2. Enable ADB debugging on your device via USB or wirelessly, then connect it to your computer.
2. For emulator, you don't need screen mirror, and emulator generally default to enabling ADB wired debugging. So this is the best way for game, I think.
4. Navigate to the Settings page -> Mask Settings, set the width and height of the mask to the same ratio of the device screen size and ensure that the mask size is appropriate.
5. Navigate to the Mask page where you can see a transparent mask. Next, adjust and move your emulator window or screen mirroring window to align the displayed content area with the transparent mask area.
6. Navigate to the Key mapping page and switch or edit the key mapping configs.
If you are interested in this project, you are welcome to submit pull request or issue.But my time and energy is limited, so I may not be able to deal with it all.
