Encodable trait objects

[steveklabnik]

Issue by kmcallister
Monday Dec 15, 2014 at 10:30 GMT

For earlier discussion, see rust-lang/rust#19874

This issue was labelled with: A-an-interesting-project, E-hard in the Rust repository

---

<kmc> i want a world where I can send a Box<Encodable + FnOnce(A) -> B> across the network and call it on the other end
<Yurume> kmc: with a built-in AEAD protocol...

[Rufflewind]

I think it’s possible to do this without making any changes to how trait objects are handled. It also doesn't require any form of trait polymorphism.

We would start with a magical built-in trait (call it TyConst for now):

// This represents a mapping from a type T to some DATA: Self.
// (The ordering here is important to avoid causing orphan issues.)
pub trait TyConst<T: ?Sized + 'static>: Sized {
    const DATA: Self;
}

// Get the key associated with TyConst<T> for D.  Uses of this function
// determine what goes into the type constant table for D.
pub fn get_ty_const_key<D: TyConst<T>, T: ?Sized + 'static>() -> usize;

// Get the DATA field in the impl for the type that matches the given key.
// If no such impl is found, returns None.
pub fn get_ty_const<D>(key: usize) -> Option<D>;

There is no way to define get_ty_const portably in today’s Rust. In fact, the function get_ty_const can't be generated until the entire program has been compiled because it requires knowing every instantiation of get_ty_const_key in the entire program!

During translation phase, the compiler would have to sift through all instantiations of get_ty_const_key and create a global table [D; _] for every D. In each table, an entry containing the matching DATA is inserted for every instantiation of get_ty_const_key<D, T>. The get_ty_const_key function is set to return the index of this entry, whereas get_ty_const simply retrieves the entry at index key from the table that corresponds to D.

Using this one trait, and with the help of a serialization package like serde, we can build a library that decodes arbitrary function-like objects:

// public interface

pub trait EncodableFn: Encodable + 'static {
    fn ty_const_key(&self) -> usize { get_ty_const_key::<FnDecoder, Self>() }
    fn call(self: Box<Self>);
}

impl Encodable for Box<EncodableFn> {
    fn encode(&self, e: &mut Encoder) -> Result<(), Error> {
        self.ty_const_key().encode(e)?;
        self.encode(e)
    }
}

impl Decodable for Box<EncodableFn> {
    fn decode(d: &mut Decoder) -> Result<Self, Error> {
        let key = Decodable::decode(d)?;
        let FnDecoder(decode) = get_ty_const(key).ok_or(Error)?;
        decode(d)
    }
}

// implementation details

trait DecodableFn: Decodable + EncodableFn {
    fn decode_fn(d: &mut Decoder) -> Result<Box<EncodableFn>, Error> {
        Self::decode(d).map(|x| Box::new(x) as Box<EncodableFn>)
    }
}

impl<T: Decodable + EncodableFn> DecodableFn for T {}

struct FnDecoder(fn(&mut Decoder) -> Result<Box<EncodableFn>, Error>);

impl<T: Decodable + EncodableFn + 'static> TyConst<T> for FnDecoder {
    const DATA: FnDecoder = FnDecoder(T::decode_fn);
}

On the users’ side, if they want to support deserialization for a custom data type Foo, they would write:

#[derive(Encodable, Decodable)]
struct Foo(Vec<u8>);

impl EncodableFn for Foo {
    fn call(self: Box<Self>) {
        println!("Foo has been called with {:?}!", self.0);
    }
}

Edit: refine idea a bit more and remove unnecessary dependence on TypeId.
Edit 2: one problem: for an incomplete program (i.e. shared lib) how would get_ty_const_key() be defined?
Edit 3: Okay here's a concrete implementation of the above idea using stable Rust plus a lot of highly questionable low-level hacks: https://crates.io/crates/detrojt

[alecmocatta]

I've taken a stab at this with rust-lang/rust#66113