Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Gamepad/keyboard navigation support #239

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

Conversation

0neGal
Copy link
Owner

@0neGal 0neGal commented Jun 14, 2024

This PR aims to add support for navigating the UI with the keyboard, but more importantly a gamepad aka a controller. While it is far from complete, the bulk of the work is here, and works quite well. The selection may still move slightly erratically in situations that have been untested, especially after changing popups and scrolling, but that's one of the things I'm intending to deal with before merging.

viper.mp4

TODO

  • Base functional navigation
  • Possible performance improvements
  • Support various controllers/gamepads
    • SteamDeck
      • While there are certainly other controllers available, and they're important, I have been able to test with the few controllers I do posses myself, I don't happen to have a SteamDeck at this very moment, so I can't confirm whether support works as intended, calling @GeckoEidechse to hopefully help with this
  • Disabled switches are still able to be switched
  • Settings popup when clicking center cluster buttons
  • Switching tabs/pages with bumper buttons or Q and E on keyboard
  • Fix selection bounds not updating if selection element bounds update
  • Fix selections staying selected even after they're no longer visible
  • Make back button only close the last opened popup, and not all popups
  • Proper scrolling support for release notes page and Thunderstore webview popup
  • Fix gamepad buttons running repeatedly, instead of only once per button down press
  • Return selection to the element that was selected before a popup was opened, when said popup is closed

TESTING BUILDS

Linux: here
Windows: here

Far from complete, but this does the bulk of the work, the rest is just
fixing places where the selection moves in weird ways after doing some
things, and overall improving the look and feel of it.
0neGal added a commit that referenced this pull request Jun 15, 2024
This will be particularly useful for #239
0neGal added a commit that referenced this pull request Jun 15, 2024
This will be particularly useful for #239
@0neGal 0neGal added the enhancement New feature or request label Jun 19, 2024
If a selected element has its bounds changed, so should the
`#selection`, this makes it so it now does that.
@0neGal 0neGal marked this pull request as ready for review June 25, 2024 10:12
@0neGal
Copy link
Owner Author

0neGal commented Jun 25, 2024

This PR is pretty much done, and the only thing that's really missing is confirming other gamepads are supported. More importantly, whether the SteamDeck works as intended, i.e the gamepad is detected, and the buttons are correct (A button = Accept, B button = Back, etc)

I would therefore love if someone with a SteamDeck (@GeckoEidechse), or even other gamepads could test this. Builds are available at the bottom of the initial PR description.

@GeckoEidechse
Copy link
Contributor

My schedule is too busy atm to test this, sorry :/

@0neGal
Copy link
Owner Author

0neGal commented Jun 25, 2024

Ah I see, I'll likely keep the PR open for a lil while, due to being busy myself, if you do get the chance to test it lmk. Even if not, I will likely merge this and make a release at some point when I get around to it. :)

@barnabwhy
Copy link

barnabwhy commented Jul 10, 2024

I think this is awesome, but I'm not sure if your sampled approach to finding elements to navigate to is the best way of doing it. Like if it performs well and does what you want, that's swag.

However, these are my thoughts after making something similar for TFORevive:
Repeatedly calling document.elementsFromPoint is really expensive and it caused stuttering in tfor, so instead we went for an approach of making a list of all the elements that are in an area and with this list of elements we can just for loop through them to find the closest.

fyi i made it in tfor after seeing this pr and getting inspiration from it :)

@0neGal
Copy link
Owner Author

0neGal commented Jul 10, 2024

I've done similar things in other projects before, it's effectively just caching the results, and I did intend to sorta get around to doing something

My idea originally was to make a wrapper for elementsFromPoint() that effectively just caches the values, i.e 0,0 gets a lil cache, and we only actually run elementsFromPoint() if this cache is slightly outdated (or an element was created), i.e if its older than 100-1000ms old or similar.

I did know this wasn't very performant while writing it, but just never got around to fixing that :p, mostly due to how it could pretty easily be fixed without requiring major refactoring.

I also considered something like a resize observer for invalidating the cache etc...

I'll probably either implement it whenever I get the time, then merge, or merge, then fix it in post, so to speak... (possibly just the prior)

@Jan200101
Copy link
Contributor

I would therefore love if someone with a SteamDeck

Do you still need someone with a SteamDeck to test this?
I have SteamDeck LED with stock SteamOS which should be great for testing purposes.

@0neGal
Copy link
Owner Author

0neGal commented Aug 2, 2024

I do yup, if you're able to test it then that'd be much appreciated! :)

@Jan200101
Copy link
Contributor

Works just fine on Fedora 40 x86_64 with Haute42 (Generic Xinput), Xbox One Controller (Wired and Wireless Dongle) but the situation is a bit different on the steam deck:

by default the Steam Deck uses a desktop layout (this can be toggled by holding the start button for a few seconds)

the desktop mode mostly works:
a = enter
b = esc
dpad & left stick = arrow keys

the L1 and R1 bumpers are bound to Alt and Ctrl by default (afaik) so that doesn't quite work, but you can navigate there via the dpad regardless.

One other thing I have noticed that I found a bit annoying is that pressing left on the launch button put me on the navigation bar, not on the sidebar

@0neGal
Copy link
Owner Author

0neGal commented Aug 3, 2024

the L1 and R1 bumpers are bound to Alt and Ctrl by default (afaik) so that doesn't quite work, but you can navigate there via the dpad regardless.

I guess I could map those to the action the bumpers normally have, I don't think this would have any negative effects, and trying to just detect whether it's a SteamDeck first would be clunky and there's likely other devices that'd want this...

One other thing I have noticed that I found a bit annoying is that pressing left on the launch button put me on the navigation bar, not on the sidebar

Yeah, the UI navigation is made to be adaptive/agnostic, so there's no hints from the UI on where to go, so it instead tries to guess where the right place to go is, might still need some tuning... :p

@0neGal
Copy link
Owner Author

0neGal commented Aug 15, 2024

SteamDeck bumper buttons should now work in the desktop layout (hopefully?), although, I did find it weird that L1 is Alt and L2 is Ctrl and not reverse, due to their typical location on most keyboards, so it's not as intuitively useful outside of the SteamDeck, but that's a minor thing.

@Jan200101
Copy link
Contributor

turns out, I did a goof here and mixed itu p
L1/LB is Left Ctrl and R1/RB is Left Alt
I've still gone ahead and tested it and it works without a hitch (ignoring that it goes in the wrong direction)

@0neGal
Copy link
Owner Author

0neGal commented Aug 15, 2024

turns out, I did a goof here and mixed itu p

whoopsie, welp, amended and should be fixed now...

@Jan200101
Copy link
Contributor

re-tested, works perfectly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants