Feature: External/Manual Inputs

[formula1]

Looks like a cool project. One thing I noticed in the instance methods is that there are no manual input methods. Granted, keyboard and gamepad may be enough to control/play the game but a developer might want to have "on screen" controls for things like mobile devices. In addition, a player might want to customize the inputs of a gamepad to their liking or have something like a turbo (when a player doesn't want to mash a button) or toggle (so you don't have to hold down the run button in mario)

If you want to point me in the direction of where the inputs are handled I wouldn't mind taking a look at what I can do

[arianrhodsandlot]

Thank you for your interest!

I assume you are talking about two topics: input mapping and input simulating.

input mapping

RetroArch, the emulator program that Nostalgist.js is using under the hood, supports input mapping by default. So the only thing we need to do is configure it. We can use retroarchConfig like this:

await Nostalgist.launch({,
  core,
  rom,
  retroarchConfig: {
    input_player1_up: 'w',
    input_player1_left: 'a',
    input_player1_down: 's',
    input_player1_right: 'd',
  }
})

Then we can use WASD as a D-pad.

Related code:

• https://github.com/libretro/RetroArch/blob/master/retroarch.cfg

• nostalgist/src/types/retroarch-config/skeleton.ts

  Lines 845 to 1131 in cbb6a03

  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_a: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_b: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_y: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_x: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_start: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_select: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_l: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_r: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_left: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_right: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_up: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_down: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_l2: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_r2: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_l3: string
  	/**
  	* Keyboard input. Will recognize letters ("a" to "z") and the following special keys (where "kp_"
  	* is for keypad keys):
  	*
  	* left, right, up, down, enter, kp_enter, tab, insert, del, end, home,
  	* rshift, shift, ctrl, alt, space, escape, add, subtract, kp_plus, kp_minus,
  	* f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12,
  	* num0, num1, num2, num3, num4, num5, num6, num7, num8, num9, pageup, pagedown,
  	* keypad0, keypad1, keypad2, keypad3, keypad4, keypad5, keypad6, keypad7, keypad8, keypad9,
  	* period, capslock, numlock, backspace, multiply, divide, print_screen, scroll_lock,
  	* tilde, backquote, pause, quote, comma, minus, slash, semicolon, equals, leftbracket,
  	* backslash, rightbracket, kp_period, kp_equals, rctrl, ralt
  	*
  	* Keyboard input, Joypad and Joyaxis will all obey the "nul" bind, which disables the bind completely,
  	* rather than relying on a default.
  	*/
  	input_player1_r3: string

Additionally, some "core" may provide a certain configuation for "turbo". Taking the NES emulator fceumm as an example,

await Nostalgist.launch({,
  core: 'fceumm',
  rom,
  retroarchCoreConfig: {
    fceumm_turbo_enable: 'Both',
  }
})

Then "x" and "y" on XBox controllers or corresponding keys on keyboard(A and S by default) will be "turbo b" and "turbo a"

Related document: libretro/docs/docs/library/fceumm.md

Considering that not all users are familiar with how to configure RetroArch, perhaps I could add some sections on this topic to the website(seems that it's going to take a lot of effort... not sure how many people would like to dive into such depth), or at least some links.

input simulating

To be honest, I have no idea about how to deal with this problem currently. User inputs are handled by RetroArch directly. I'm not sure will firing DOM keypress events manually work or not, but I'll try this when I have time.

For reference, EmulatorJS, another similar library seems to support this feature, through a non official build of RetroArch.

[arianrhodsandlot]

After trying for some time, I discovered that we can simulate keyboard input by invoking a method on Emscripten Module like this:

const nostalgist = await Nostalgist.launch(/* options */)

const Module = nostalgist.getEmscriptenModule()

// press some key for 10ms
Module.dynCall_iiii(471, 2, 10474576, 9909248)
await new Promise((resolve) => setTimeout(resolve, 10))
Module.dynCall_iiii(471, 3, 10474576, 9909248)

Hmmm.. it's so hard to understand!

I think I should implement some new APIs later, to make this easier. Maybe something like nostalgist.press('xxx'). I'll keep this issue open until it's usable. Thanks for your suggestion!

[formula1]

This looks above my paygrade, how did you find that?

nostalgist.press('xxx') sounds great. It would be nice if it would be something along the lines of

nostalgist.press = (player: number, button: ButtonIdentifier, toggleValue: boolean)=>{
  // Module call here
}

An additional function regarding plugging in and unplugging controller would be nice also although that only really applies to nes/snes/genesis etc.

[arianrhodsandlot]

how did you find that?

By using devtools' "event listeners" panel and debugging an emulator's js file like fceumm_libretro.js.

An additional function regarding plugging in and unplugging controller would be nice

Do you mean gamepadconnected event? And this is for browsers with real controllers.

If you are talking about emulators, there doesn't seem to be such a thing. Abstract controllers seem to be always "connected" from emulators' perspective. Emulators read input data and update their image accordingly, that's all they do.

[arianrhodsandlot]

It's here now🚀!