feat: Add support for async/Future

[Hywan]

Quick explanation

This is a patch to add support for async/Future inside uniffi-rs. At the time of writing, it works very well with Python, Swift and Kotlin.

Any Future<Output = T> or Future<Output = Result<T, E>>, where T: FfiReturn, is supported. The code is documented and I reckon it's enough to understand the code. The main challenges are:

1. To expose a (monomorphized) API of a Future<T> with a C ABI,
2. To fit the Rust model for Future into other foreign language executors/runtimes. The Rust model is very different, but since a Future does nothing by default and must be polled by an executor, that's actually perfect! In the case of Python, the executor is asyncio, in the case of Swift, the executor is the platform itself, in the case of Kotlin, the executor is… what suspend inherits. The biggest difficulty is to be able to provide a waker defined in the foreign language to Rust.

The documentation with detailed explanations of this project lands here:

uniffi-rs/uniffi_core/src/ffi/rustfuture.rs

Lines 1 to 282 in 926135a

	//! [`RustFuture`] represents a [`Future`] that can be sent over FFI safely-ish.
	//!
	//! The [`RustFuture`] type holds an inner `Future<Output = Result<E, T>>`, and
	//! thus is parameterized by `T` and `E`. On the `RustFuture` type itself, there
	//! is no constraint over those generic types (constraints are present in the
	//! [`uniffi_rustfuture_poll`] function, where `T: FfiReturn`, see later
	//! to learn more). Every function or method that returns a `Future` must
	//! transform the result into a `Result`. This is done by the procedural
	//! macros automatically: `Future<Output = T>` is transformed into `RustFuture<T,
	//! Infallible>`, and `Future<Output = Result<T, E>>` is transformed into
	//! `RustFuture<T, E>`.
	//!
	//! This type may not be instantiated directly, but _via_ the procedural macros,
	//! such as `#[uniffi::export]`. A `RustFuture` is created, boxed, and then
	//! manipulated by (hidden) helper functions, resp. [`uniffi_rustfuture_poll`]
	//! and [`uniffi_rustfuture_drop`]. Because the `RustFuture` type contains a
	//! generic parameters `T` and `E`, the procedural macros will do a
	//! monomorphisation phase so that all the API has all their types statically
	//! known.
	//!
	//! # The big picture
	//!
	//! This section will explain how the entire workflow works.
	//!
	//! Let's consider the following Rust function:
	//!
	//! ```rust,ignore
	//! #[uniffi::export]
	//! async fn hello() -> bool {
	//! true
	//! }
	//! ```
	//!
	//! In Rust, this `async fn` syntax is strictly equivalent to:
	//!
	//! ```rust,ignore
	//! #[uniffi::export]
	//! fn hello() -> impl Future<Output = bool> { /* … */ }
	//! ```
	//!
	//! Once this is understood, it becomes obvious that an `async` function
	//! returns a `Future`.
	//!
	//! This function will not be modified, but new functions with a C ABI will be
	//! created, as such:
	//!
	//! ```rust,ignore
	//! /// The `hello` function, as seen from the outside. It returns a `Future`, or
	//! /// more precisely, a `RustFuture` that wraps the returned future.
	//! #[no_mangle]
	//! pub extern "C" fn _uniffi_hello(
	//! call_status: &mut ::uniffi::RustCallStatus
	//! ) -> Option<Box<::uniffi::RustFuture<bool, ::std::convert::Infallible>>> {
	//! ::uniffi::call_with_output(call_status, || {
	//! Some(Box::new(::uniffi::RustFuture::new(async move {
	//! Ok(hello().await)
	//! })))
	//! })
	//! }
	//! ```
	//!
	//! This function returns an `Option<Box<RustFuture<T, E>>`:
	//!
	//! * Why an `Option`? Because in case of an error, it must return a default
	//! value, in this case `None`, otherwise `Some`. If we were returning `Box`
	//! directly, it would leak a pointer.
	//!
	//! * Why `Box`? Because Rust doesn't own the future, it's passed to the
	//! foreign language, and the foreign language is responsible to manage it.
	//!
	//! * Finally, this function calls the real Rust `hello` function. It
	//! transforms its result into a `Result` if it's needed.
	//!
	//! The second generated function is the _poll function_:
	//!
	//! ```rust,ignore
	//! /// The function to poll the `RustFuture` returned by `_uniffi_hello`.
	//! #[no_mangle]
	//! pub extern "C" fn _uniffi_hello_poll(
	//! future: Option<&mut ::uniffi::RustFuture<bool, ::std::convert::Infallible>>,
	//! waker: Option<NonNull<::uniffi::RustFutureForeignWakerFunction>>,
	//! waker_environment: *const ::uniffi::RustFutureForeignWakerEnvironment,
	//! polled_result: &mut <bool as ::uniffi::FfiReturn>::FfiType,
	//! call_status:: &mut ::uniffi::RustCallStatus,
	//! ) -> bool {
	//! ::uniffi::ffi::uniffi_rustfuture_poll(future, waker, waker_environment, polled_result, call_status)
	//! }
	//! ```
	//!
	//! Let's analyse this function because it's an important one:
	//!
	//! * First off, this _poll FFI function_ forwards everything to
	//! [`uniffi_rustfuture_poll`]. The latter is generic, while the former has been
	//! monomorphised by the procedural macro.
	//!
	//! * Second, it receives the `RustFuture` from `_uniffi_hello` as an
	//! `Option<&mut RustFuture<_>>`. It doesn't take ownership of the `RustFuture`!
	//! It borrows it (mutably). It's wrapped inside an `Option` to check whether
	//! it's a null pointer or not; it's defensive programming here, `null` will
	//! make the Rust code to panic gracefully.
	//!
	//! * Third, it receives a _waker_ as a pair of a _function pointer_ plus its
	//! _environment_, if any; a null pointer is purposely allowed for the environment.
	//! This waker function lives on the foreign language side. We will come back
	//! to it in a second.
	//!
	//! * Fourth, it receives an in-out `polled_result` argument, that is filled with the
	//! polled result if the future is ready.
	//!
	//! * Firth, the classical `call_status`, which is part of the calling API of `uniffi`.
	//!
	//! * Finally, the function returns `true` if the future is ready, `false` if pending.
	//!
	//! Please don't forget to read [`uniffi_rustfuture_poll`] to learn how
	//! `polled_result` + `call_status` + the returned bool type are used to indicate
	//! in which state the future is.
	//!
	//! So, everytime this function is called, it polls the `RustFuture` after
	//! having reconstituted a valid [`Waker`] for it. As said earlier, we will come
	//! back to it.
	//!
	//! The last generated function is the _drop function_:
	//!
	//! ```rust,ignore
	//! #[no_mangle]
	//! pub extern "C" fn _uniffi_hello_drop(
	//! future: Option<Box<::uniffi::RustFuture<bool, ::std::convert::Infallible>>>,
	//! call_status: &mut ::uniffi::RustCallStatus
	//! ) {
	//! ::uniffi::ffi::uniffi_rustfuture_drop(future, call_status)
	//! }
	//! ```
	//!
	//! First off, this _drop function_ is responsible to drop the `RustFuture`. It's
	//! clear by looking at its signature: It receives an `Option<Box<RustFuture<_>>>`,
	//! i.e. it takes ownership of the `RustFuture` _via_ `Box`!
	//!
	//! Similarly to the _poll function_, it forwards everything to
	//! [`uniffi_rustfuture_drop`], which is the generic version of the monomorphised _drop
	//! function_.
	//!
	//! ## How does `Future` work exactly?
	//!
	//! A [`Future`] in Rust does nothing. When calling an async function, it just
	//! returns a `Future` but nothing has happened yet. To start the computation,
	//! the future must be polled. It returns [`Poll::Ready(r)`][`Poll::Ready`] if
	//! the result is ready, [`Poll::Pending`] otherwise. `Poll::Pending` basically
	//! means:
	//!
	//! > Please, try to poll me later, maybe the result will be ready!
	//!
	//! This model is very different than what other languages do, but it can actually
	//! be translated quite easily, fortunately for us!
	//!
	//! But… wait a minute… who is responsible to poll the `Future` if a `Future` does
	//! nothing? Well, it's _the executor_. The executor is responsible _to drive_ the
	//! `Future`: that's where they are polled.
	//!
	//! But… wait another minute… how does the executor know when to poll a [`Future`]?
	//! Does it poll them randomly in an endless loop? Well, no, actually it depends
	//! on the executor! A well-designed `Future` and executor work as follows.
	//! Normally, when [`Future::poll`] is called, a [`Context`] argument is
	//! passed to it. It contains a [`Waker`]. The [`Waker`] is built on top of a
	//! [`RawWaker`] which implements whatever is necessary. Usually, a waker will
	//! signal the executor to poll a particular `Future`. A `Future` will clone
	//! or pass-by-ref the waker to somewhere, as a callback, a completion, a
	//! function, or anything, to the system that is responsible to notify when a
	//! task is completed. So, to recap, the waker is _not_ responsible for waking the
	//! `Future`, it _is_ responsible for _signaling_ the executor that a particular
	//! `Future` should be polled again. That's why the documentation of
	//! [`Poll::Pending`] specifies:
	//!
	//! > When a function returns `Pending`, the function must also ensure that the
	//! > current task is scheduled to be awoken when progress can be made.
	//!
	//! “awakening” is done by using the `Waker`.
	//!
	//! ## Awaken from the foreign language land
	//!
	//! Our _poll function_ receives a waker function pointer, along with a waker
	//! environment. We said that the waker function lives on the foreign language
	//! side. That's really important. It cannot live inside Rust because Rust
	//! isn't aware of which foreign language it is run from, and thus doesn't know
	//! which executor is used. It is UniFFI's job to write a proper foreign waker
	//! function that will use the native foreign language's executor provided
	//! by the foreign language itself (e.g. `Task` in Swift) or by some common
	//! libraries (e.g. `asyncio` in Python), to ask to poll the future again.
	//!
	//! We expect the waker to be the same per future. This property is not
	//! checked, but the current foreign language implementations provided by UniFFI
	//! guarantee that the waker is the same per future everytime. Changing the
	//! waker for the same future leads to undefined behaviours, and may panic at some
	//! point or leak data.
	//!
	//! The waker must live longer than the `RustFuture`, so its lifetime's range
	//! must include `_uniffi_hello()` to `_uniffi_hello_drop()`, otherwise it leads to
	//! undefined behaviours.
	//!
	//! ## The workflow
	//!
	//! 1. The foreign language starts by calling the regular FFI function
	//! `_uniffi_hello`. It gets an `Option<Box<RustFuture<_>>>`.
	//!
	//! 2. The foreign language immediately polls the future by using the `_uniffi_hello_poll`
	//! function. It passes a function pointer to the waker function, implemented
	//! inside the foreign language, along with its environment if any.
	//!
	//! - Either the future is ready and computes a value, in this case the _poll
	//! function_ will lift the value and will drop the future with the _drop function_
	//! (`_uniffi_hello_drop`),
	//!
	//! - or the future is pending (not ready), and is responsible to register
	//! the waker (as explained above).
	//!
	//! 3. When the waker is called, it calls the _poll function_, so we basically jump
	//! to point 2 of this list.
	//!
	//! There is an important subtlety though. Imagine the following Rust code:
	//!
	//! ```rust,ignore
	//! let mut shared_state: MutexGuard<_> = a_shared_state.lock().unwrap();
	//!
	//! if let Some(waker) = shared_state.waker.take() {
	//! waker.wake();
	//! }
	//! ```
	//!
	//! This code will call the waker. That's nice and all. However, when the waker
	//! function is called by `waker.wake()`, this code above has not returned yet.
	//! And the waker function, as designed so far, will call the _poll function_
	//! of the Rust `Future` which… may use the same lock (`a_shared_state`),
	//! which is not released yet: there is a dead-lock! Rust is not responsible of
	//! that, kind of. Rust **must ignore how the executor works**, all `Future`s
	//! are executor-agnostic by design. To avoid creating problems, the waker
	//! must “cut” the flow, so that Rust code can continue to run as expected, and
	//! after that, the _poll function_ must be called.
	//!
	//! Put differently, the waker function must call the _poll function_ _as
	//! soon as possible_, not _immediately_. It actually makes sense: The waker
	//! must signal the executor to schedule a poll for a specific `Future` when
	//! possible; it's not an inline operation. The implementation of the waker
	//! must be very careful of that.
	//!
	//! With a diagram (because this comment would look so much clever with a diagram),
	//! it looks like this:
	//!
	//! ```text
	//! ┌────────────────────┐
	//! │ │
	//! │ Calling hello │
	//! │ │
	//! └─────────┬──────────┘
	//! │
	//! ▼ fn waker ◄──┐
	//! ┌────────────────────────────────┐ │
	//! │ │ │
	//! │ Ask the executor to schedule │ │
	//! │ this as soon as possible │ │
	//! │ │ │
	//! │ ┌──────────────────────────┐ │ │
	//! │ │ │ │ │
	//! │ │ Polling the RustFuture │ │ │
	//! │ │ Pass pointer to waker ──┼──┼──┘
	//! │ │ │ │
	//! │ └────────────┬─────────────┘ │
	//! │ │ │
	//! └───────────────┼────────────────┘
	//! │
	//! ▼
	//! ┌──── The future is ─────┐
	//! │ │
	//! Ready Pending
	//! │ │
	//! ▼ ▼
	//! ┌───────────────┐ ┌──────────────────────┐
	//! │ Lift result │ │ Nothing │
	//! │ Have fun │ │ Let's wait for the │
	//! └───────────────┘ │ waker to be called │
	//! └──────────────────────┘
	//! ```
	//!
	//! That's all folks!

Overview

Let's start with the following Rust code:

#[uniffi::export]
async fn say_after(secs: u8, who: String) -> String {
    TimerFuture::new(Duration::from_secs(secs.into())).await;

    format!("Hello, {who}!")
}

Nothing special except that the async function is annotated with #[uniffi::export].

In Python

The function definition

async def say_after(secs, who)

How to use it

from uniffi_futures import say_after

async def main():
    result_alice = await say_after(2, 'Alice')
    result_bob = await say_after(3, 'Bob')

    print(f'result_alice: {result_alice}')
    print(f'result_bob: {result_bob}')

asyncio.run(main())

This code awaits on futures sequentially. It returns after $2 + 3 = 5$ seconds. However, since we generate proper Python Future-ish (we use our own type, but it's 100% compatible with the standard asyncio.Future), we can create asyncio.Tasks to run them concurrently:

async def main():
    alice = asyncio.create_task(say_after(2, 'Alice'))
    bob = asyncio.create_task(say_after(3, 'Bob'))

    result_alice = await alice
    result_bob = await bob

    print(f'result_alice: {result_alice}')
    print(f'result_bob: {result_bob}')

asyncio.run(main())

This code runs in $\max(2, 3) = 3$ seconds.

In Swift

Now, let's jump on Swift! Contrary to Python where we implement our own Future-ish type, in Swift, the generated functions and methods are 100% native standard async functions.

The function definition

public func sayAfter(secs: UInt8, who: String) async -> String

How to use it

Hint: To run async code, we must be inside a Task.

import uniffi_futures

Task {
    let result_alice = await sayAfter(secs: 2, who: "Alice")
    let result_bob = await sayAfter(secs: 3, who: "Bob")

    print("result_alice: \(result_alice)")
    print("result_bob: \(result_bob)")
}

This code awaits on futures sequentially. It returns after $2 + 3 = 5$ seconds. But because we generate native async functions, it fits well with the native language concurrency constructions (like async let):

Task {
    async let alice = sayAfter(secs: 2, who: "Alice")
    async let bob = sayAfter(secs: 2, who: "Bob")

    let (result_alice, result_bob) = await (alice, bob)
    print("result_alice: \(result_alice)")
    print("result_bob: \(result_bob)")
}

This code runs in $\max(2, 3) = 3$ seconds.

In Kotlin

OK, Kotlin now! In the manner of Swift, the generated async functions, aka suspended functions use everything native from the Kotlin language.

The function definition

suspend fun sayAfter(secs: UByte, who: String): String

How to use it

import uniffi.futures.*

fun main() = runBlocking {
    val resultAlice = sayAfter(2U, "Alice")
    val resultBob = sayAfter(2U, "Bob")

    print("resultAlice: ${resultAlice}")
    print("resultBob: ${resultBob}")
}

You can't really tell when a call to a function is sync or async in Kotlin, but trust me, they are async.

This code awaits on futures sequentially. It returns after $2 + 3 = 5$ seconds. Let's execute them concurrently with async:

val alice = async { sayAfter(2U, "Alice") }
val bob = async { sayAfter(3U, "Bob") }

println("alice: ${alice.await()}")
println("bob: ${bob.await()}")

This code runs in $\max(2, 3) = 3$ seconds.

Deeper look

Generated code with annotations.

In Python

async def say_after(secs,who):
    # The arguments.
    secs = int(secs)
    who = who
    
    # The `RustFuture` is created here, but not polled!
    rust_future = rust_call(_UniFFILib._uniffi_uniffi_futures_say_after_8d91,
        FfiConverterUInt8.lower(secs),
        FfiConverterString.lower(who))
    future = None

    # The “future” logic.
    def trampoline() -> (FuturePoll, any):
        nonlocal rust_future

        # Prepare to receive the polled result if the future is ready.
        polled_result = RustBuffer()
        polled_result_ref = ctypes.byref(polled_result)

        # Let's call the poll function!
        is_ready = rust_call(
            _UniFFILib._uniffi_uniffi_futures_say_after_8d91_poll,
            rust_future, # the rust future
            future._future_ffi_waker(), # the waker
            ctypes.c_void_p(), # no waker environment
            polled_result_ref, # the polled result ref
        )

        if is_ready is True:
            # Lift the result from Rust to Python
            result = FfiConverterString.lift(polled_result)

            return (FuturePoll.DONE, result)
        else:
            return (FuturePoll.PENDING, None)

    # Let's use our `Future` implementation, à la `asyncio.Future`
    future = Future(trampoline)
    future.init()

    # Let's await on the future.
    result = await future

    # Let's drop the `RustFuture`.
    rust_call(_UniFFILib._uniffi_uniffi_futures_say_after_8d91_drop, rust_future)

    return result

In Swift

// This is an environment that will be used as the waker's environment.
private class _UniFFI_SayAfter_Env {
    // The `RustFuture`.
    var rustFuture: OpaquePointer
    // We use “continuation” from Swift to get async support.
    var continuation: CheckedContinuation<String, Never>

    init(rustyFuture: OpaquePointer, continuation: CheckedContinuation<String, Never>) {
        rustFuture = rustyFuture
        self.continuation = continuation
    }

    // When the waker's environment is dropped, the `RustFuture` is dropped.
    deinit {
        try! rustCall {
            _uniffi_uniffi_futures_say_after_8d91_drop(self.rustFuture, $0)
        }
    }
}

// This is the waker function associated for the `say_after` function.
private func _UniFFI_SayAfter_waker(raw_env: UnsafeMutableRawPointer?) {
    // Schedule the poll of the `RustFuture`. It's not done immediately, but as soon as possible.
    Task {
        let env = Unmanaged<_UniFFI_SayAfter_Env>.fromOpaque(raw_env!)
        let env_ref = env.takeUnretainedValue()
        // Prepare to receive the polled result.
        let polledResult = UnsafeMutablePointer<RustBuffer>.allocate(capacity: 1)

        // Let's call the poll function!
        let isReady = try! rustCall {
            _uniffi_uniffi_futures_say_after_8d91_poll(
                env_ref.rustFuture, // the `RustFuture`
                _UniFFI_SayAfter_waker, // the waker function
                env.toOpaque(), // the waker's environment
                polledResult, // the polled result reference
                $0 // the call status
            )
        }

        if isReady {
            // When the future is ready, we can resume the execution of the
            // continuation with the lifted value from Rust to Swift.
            env_ref.continuation.resume(returning: try! FfiConverterString.lift(polledResult.move()))

            // Clean everything.
            polledResult.deallocate()
            env.release()
        }
    }
}

// Finally, the `say_after` implementation.
public func sayAfter(secs: UInt8, who: String) async -> String {
    // Let's create the `RustFuture`.
    let future = try! rustCall {
        _uniffi_uniffi_futures_say_after_8d91(
            FfiConverterUInt8.lower(secs),
            FfiConverterString.lower(who), $0
        )
    }

    // Let's create a continuation…
    return await withCheckedContinuation { continuation in
        // … and let's prepare the waker's environment
        let env = Unmanaged.passRetained(_UniFFI_SayAfter_Env(rustyFuture: future, continuation: continuation))
        // … to finally call the waker function (that will poll the future).
        _UniFFI_SayAfter_waker(raw_env: env.toOpaque())
    }
}

In Kotlin

// The `say_after` implementation, it's self-contained.
suspend fun `sayAfter`(`secs`: UByte, `who`: String): String {
    // The waker “function” (a class having a `callback` method in JNA is an FFI function callback).
    class Waker : RustFutureWaker {
        private val lock = Semaphore(1)

        override fun callback(envCStructure: RustFutureWakerEnvironmentCStructure?) {
            if (envCStructure == null) {
                return;
            }

            val hash = envCStructure.hash
            val env = _UniFFILib.FUTURE_WAKER_ENVIRONMENTS.get(hash)

            // Check whether the future is still not resolved
            if (env == null) {
                return
            }

            // Schedule the poll of the `RustFuture`. It's not done immediately, but as soon as possible.
            env.coroutineScope.launch {
                // Allow only one polling at a time
                lock.withPermit {
                    // Check if the future hasn't been resolved since last call.
                    if (!_UniFFILib.FUTURE_WAKER_ENVIRONMENTS.containsKey(hash)) {
                        return@withPermit
                    }

                    // Like Swift, we use continuation in Kotlin to get async support.
                    @Suppress("UNCHECKED_CAST")
                    val continuation = env.continuation as Continuation<String>
                    // Let's prepare the polled result reference.
                    val polledResult = RustBufferByReference()
    
                    try {
                        // Let's poll the `RustFuture`!
                        val isReady = rustCall() { _status ->
                            _UniFFILib.INSTANCE._uniffi_uniffi_futures_say_after_8d91_poll(
                                env.rustFuture, // the `RustFuture`
                                env.waker, // the waker function
                                env.selfAsCStructure, // the waker's environment
                                polledResult, // the polled result reference
                                _status // the call status
                            )
                        }
    
                        if (isReady) {
                            // if it's ready, let's resume the continuation with the lifted result from Rust to Kotlin
                            continuation.resume(
                                FfiConverterString.lift(polledResult.getValue())
                            )
    
                            // and let's drop the `RustFuture`
                            _UniFFILib.FUTURE_WAKER_ENVIRONMENTS.remove(hash)
                            rustCall() { _status ->
                                _UniFFILib.INSTANCE._uniffi_uniffi_futures_say_after_8d91_drop(env.rustFuture, _status)
                            }
                        }
                    } catch (exception: Exception) {
                        // try-catch is always present in Kotlin, while it's on-demand in Swift
                        continuation.resumeWithException(exception)
    
                        // the same cleaning dance
                        _UniFFILib.FUTURE_WAKER_ENVIRONMENTS.remove(hash)
                        rustCall() { _status ->
                            _UniFFILib.INSTANCE._uniffi_uniffi_futures_say_after_8d91_drop(env.rustFuture, _status)
                        }
                    }
            }
        }
    }

    val result: String

    // Now, we can do the serious business
    coroutineScope {
        // suspend the coroutine
        result = suspendCoroutine<String> { continuation ->
            // let's call the `RustFuture`
            val rustFuture = rustCall() { _status ->
                _UniFFILib.INSTANCE._uniffi_uniffi_futures_say_after_8d91(FfiConverterUByte.lower(`secs`), FfiConverterString.lower(`who`), _status)
            }

            // let's create the waker's environment
            val env = RustFutureWakerEnvironment(rustFuture, continuation, Waker(), RustFutureWakerEnvironmentCStructure(), this /* this = the coroutine scope */)
            val envHash = env.hashCode()
            env.selfAsCStructure.hash = envHash

            // save the waker's environment to avoid GC' surprises
            _UniFFILib.FUTURE_WAKER_ENVIRONMENTS.put(envHash, env)

            // create the waker and…
            val waker = Waker()
            // … call it!
            waker.callback(env.selfAsCStructure)
        }
    }

    return result
}

Progress

• RustFuture<T>
  • Can be polled (and can return any value T)
  • Can be dropped (so can be cancelled)
  • RustFuture.poll(self, waker) -> Poll<T>
  • uniffi_rustfuture_poll assistant/helper function for C ABI
  • uniffi_rustfuture_drop assistant/helper function for C ABI
• #[uniffi::export] supports Future
  • Monomorphize _poll function
  • Monomorphize _drop function
• Add Python support
  • For functions
    • With tests
  • For methods
    • With tests
• Add Swift support
  • For functions
    • With tests
  • For methods
    • With tests
• Add Kotlin support
  • For functions
    • With tests
  • For methods
    • With tests
• Ruby not planned, I don't need it, maybe later
• Support Tokio's Future
  • Use #[uniffi::export(async_runtime = "tokio")] to add the glue to make Tokio's Futures compatible with the standard std::future::Future type
  • With tests
• Support Future<Output = ()>
• Support fallible functions, i.e. Future<Output = Result<T, E>>
  • Add Python support
    • With tests
  • Add Swift support
    • With tests
  • Add Kotlin support
    • With tests
• Test with more complex types returned by Future::poll
• Support “waker re-entry protection”
  • In Kotlin
  • In Swift
  • In Python

[Hywan]

@bendk The last blocker is the update of the Docker container image (as explained here). Can we unblock this situation? Is it possible for someone else to update this image?

[bendk]

I wish I could, but I don't have access to push the container.

It would be really nice if the CI used the Dockefile from the repo. I don't have time today, but I'll check on this tomorrow.

[rfk]

I wish I could, but I don't have access to push the container.

Is this still the "That only works if you're rfkelly; we need to figure out a better strategy for maintainership of said docker image." problem mentioned in https://github.com/mozilla/uniffi-rs/blob/main/docker/README.md? Let me know if there's something I can do to help unblock things here in the short term 😬

[bendk]

@rfk Thanks, but I think we can have a different short-term fix.

@Hywan I think the best path forward is to make these tests not run in CI. The first step there is making the other tests still pass. I made a quick commit that makes it so we only import the async libraries if there are actually async functions. I think that should make them pass again. Hopefully, just a few more steps after that:

• Make it so the futures tests don't run in CI. If you don't know a better way, we can just start with commenting out the code in fixtures/futures/tests/test_generated_bindings.rs.
• Make it so the dockerfile builds again. I'm seeing issues while downloading the swift packages. I think curl just needs o -L to fix that.
• Add checksum checks to the Dockerfile. I'd really love to see those checks before merging. That way, once badboy is ready he can build and push the ci container. I don't think we should do that before the checksums.

[Hywan]

• Add checksum checks to the Dockerfile. I'd really love to see those checks before merging. That way, once badboy is ready he can build and push the ci container. I don't think we should do that before the checksums.

Do you mean to add checksums for each JAR or stuff like that we download inside the Dockerfile-build file?

[Hywan]

So, the Docker image cannot run the Swift and Kotlin tests because:

1. Swift has introduced async/await since 5.5, and the current version used inside the Docker image is 5.2.
2. Kotlin needs kotlinx.coroutines JAR.

This PR updates the Dockerfile-build to update Swift to 5.5 (the latest version is 5.7, but let's stick on the lowest version possible), and to add the missing JAR for Kotlin.

This PR also disables the tests for Swift and Kotlin for the moment, waiting for the Docker image to be updated.

[Hywan]

CI is green \o/

[Hywan]

The RustFuture type, along with all the associated mechanisms, is tested via the Python implementation for async. Swift and Kotlin are working on our machines, just not in the CI because of the Docker image. Apart from that, everything is fine.

[bendk]

I was away for most of the week, but this is looking great. Let's merge.

[bendk]

I think the only remaining issue is to merge main in and resolve the conflicts.

[Hywan]

@bendk Ready to be merged!

[bendk]

Merged! Thanks for all the work here, I'm excited to continue to push this.

Would you want a new release to use this functionality?

[Hywan]

We are using main directly, no need for a release yet. I want to continue to improve it this week or next week. No need to rush on a new release for us :-).

[jameshoulahan]

Hi, thanks for implementing this! However, I just tried it out on main and hit some unexpected behaviour.

Suppose we have the following rust functions and we generate python bindings from them.

#[uniffi::export]
pub fn add(left: u32, right: u32) -> u32 {
    left + right
}

#[uniffi::export]
pub async fn async_add(left: u32, right: u32) -> u32 {
    left + right
}

As expected, the add function works just fine:

>>> mylib.add(1, 2)
3 

>>> mylib.add(1, 2) + mylib.add(3, 4)
10

But when calling the async_add, I'm seeing that the return type is wrapped in c_uint, which is a bit inconvenient:

>>> async def f():
...     return await mylib.async_add(1, 2)

>>> async def g():
...     val1 = await mylib.async_add(1, 2)
...     val2 = await mylib.async_add(3, 4)
...     return val1 + val2
 
>>> asyncio.run(f())
c_uint(3)

>>> asyncio.run(g())
TypeError: unsupported operand type(s) for +: 'c_uint' and 'c_uint'

I would expect the type returned after awaiting the async function to be the same as the type of the non-async variety, but it's not. Potential bug?

[Hywan]

Not a bug. An oversight ;-). Please open a proper issue, and ping me. I'll fix it.

[Sajjon]

I think there is a typo in the Swift example:

async let alice = sayAfter(secs: 2, who: "Alice")
async let bob = sayAfter(secs: 2, who: "Bob")

should be:

async let alice = sayAfter(secs: 2, who: "Alice")
async let bob = sayAfter(secs: 3, who: "Bob")

3 sec for bob, not 2, to match the description, right?

[jplatte]

Please don't post comments on old PRs, open an issue instead.