[WIP] Partially loaded recrods

[drogus]

I've done a first realy simple stab at partially loaded records and I would like to start a discussion about it. It will need to expose new APIs, so I would like to have a blessing before going to deep into implementation.

The purpose of partially loaded records is to allow creating a record from partial data and act if the missing data is accessed. For example we may have a blog with a Post model, which has 2 attributes: title and body. A body can be quite big and we don't show it on the list of posts, so to make the posts list to load turbofast, we can just return a representation containing title and load body only if a visitor goes to detailed post view.

The code could look like this:

var Post = Ember.Model.extend({
  id: attr(String),
  title: attr(String),
  body: attr(String),

  missingAttributeAccessed: function(key) {
    // reload record if we're missing any attributes
    this.reload();
  }
});

Post.loadPartially({
  id: '1',
  title: 'Really long blog post'
});

var post = Post.find('1');

post.get('title'); // => 'Really long blog post'

post.get('body'); // => null
// at this point missingAttributeAccessed is called

Current implementation ignores associations, mainly because I would like to implement belongsTo and hasMany without embedded data before I continue work on this.

Any feedback and comments on both implementation and public API is welcomed.

TODO:

• add isPartiallyLoaded computed property, so people can observe it
• add a way to indicate that the record is fully loaded
• support associations
• consider creating a public API so this can be implemented as an extension (is it worth it?)

[drogus]

I think what's missing here and should definitely be included is isPartiallyLoaded computed property. At the moment the only way to check if record is partially loaded is to use a function, which is not optimal.

[ebryn]

I think isPartiallyLoaded should probably just be a simple boolean property that we toggle manually upon loading all the data.

I also think we don't just want a loadPartially class method. You'll tend to want to do this for individual record instances I think.

[drogus]

@ebryn thanks for the input, I like that suggestion, it will make implementation much simpler. I went for the class method because I wanted to handle a scenario where you want to load records, but not necessarily materialize them. It's probably overkill and users could implement it easily by their own.

The extension could look like this:

App.Model = Ember.Model.extend({
  init: function() {
    this._super.apply(this, arguments);

    if(this.constructor._partiallyLoaded[this.get('id')]) {
      this.set('partiallyLoaded');
    }
  }
});

App.Model.reopenClass({
  loadPartially: function(hashes) {
    if (!this._partiallyLoaded) { this._partiallyLoaded = {}; }
    for (var i = 0, l = hashes.length; i < l; i++) {
      var hash = hashes[i];
      this._partiallyLoaded[hash[get(this, 'primaryKey')]] = Object.keys(hash);
    }
    this.load(hashes);
  }
});

[guilhermeaiolfi]

What if you want to split the record in 4 parts and load one part at a time? a simple isPartiallyLoaded won't be enough.

[drogus]

@guilhermeaiolfi we want to start small, so it may not handle such use case, but we're open to provide hooks for easy extension or improving it later. In order to support partially loading in a few stages you just need to have attributes list checks. I will address this later when I finish the implementation.

[jiripospisil]

How about this:

var Post = Ember.Model.extend({
  id: Ember.attr(String),
  title: Ember.attr(String),
  lead: Ember.attr(String),
  body: Ember.attr(String, {lazy: true}),

  loadLazyAttributes: {
    body: function() {
      // Delegate to the adapter, we might make this a default behavior so that
      // you don't need to define this method at all if you are fine with the
      // default.
      if (!this.get("isNew")) {
        return this.loadLazyAttribute("body");
      }
      return null;
    }
  }
});

Post.load([{id: 1, title: "Why, of course!"}]);

var post = Post.find(1);

post.get("isLoaded") // => true
post.get("isFullyLoaded") // => false, not all attributes are loaded

post.get("title.isLoaded") // => true
post.get("title") // => "Why, of course!"

// The lead attribute is missing but not marked as lazy so we assume it's null
post.get("lead.isLoaded") // => true
post.get("lead") // => null

// The body attribute is lazy so we can't really tell at this point whether the
// attribute is not loaded yet or null - search for the body key in the
// loadLazyAttributes hash.
post.get("body.isLoaded") // => false
post.get("body").then(function(body) {
  console.log(body);
});

[drogus]

I would like to make this implementation work without defining which attributes are lazy beforehand. However, I see that a lot of people have a lot of different use cases, so I'll try to provide a few simple extension points.

[drogus]

I updated the PR with new simpler code - it will check if record is partially or fully loaded every time when load() method is called (this means that all adapters using load will work out of the box). The only assumption is that all keys should be returned by the API.

I think this is a nice middle ground. Setting isPartiallyLoaded blindly in the adapter without doing any attributes check would be simpler for sure, but it would be not really useful for at least 2 use cases I used or saw in the wild. The first one is mentioned pusher and the second one is loading limited record representation also in find(id) case: initially you load partially loaded record and then you can issue a full representation with something like GET /posts/1?full=true.

If this is still too much, I would like to provide getAttr hook, so I could do this:

var Model = Ember.Model.extend({
  load: function(id, hash) {
    this._super(id, hash);

    // check if record is partially loaded by comparing attributes
  },

  getAttr: function(key) {
    // this will be called whenever an attr is being fetched
    if(this.get('isPartiallyLoaded') && this.isAttributeMissing(key)) {
      this.missingAttributeAccessed(key);
    }
  }
});

[ebryn]

I'd rather you just loop here, it'll be significantly faster.

[drogus]

After a few more discussions about this functionality with Erik and other contributors, we decided to hold on and to provide getAttr, which I submitted at #87.

Almost everyone has a slightly different use case and this doesn't seem like a widely used feature. With getAttr hook implementing this as an extension is trivial.

[ebryn]

I hope @drogus will post some example code or an extension at some point.