Tiles cache

[pronebird]

Hi,

Currently when map screen shows up I see emptiness for like 5 seconds, then the map starts zooming out slowly showing blank spaces for a second until all tiles are loaded.

At the same time bunch of XHR requests run in background, some of them hit browser cache, some get cancelled but it does not really help to provide smooth experience.

Even if all of the tiles are cached by browser it simply takes way too long for Mapbox GL to fetch images via XHR producing blank tiles when camera zooms out and moves around.

Is there any way to cache tiles in local storage, on disk or in memory to improve the smoothness and generally decrease amount of traffic?

[remster]

I use service workers + IndexedDB (where available) quite successfully.

[lucaswoj]

We do not support or plan to support an offline mode for GL JS. It will be easier to implement your own offline mode once we implement custom source types.

Your performance concerns will be addressed by #3676.

Previously #2600 #812

[pronebird]

@remster any examples of how to add caching for mapbox-gl?

[remster]

@pronebird yes. Here's the relevant bit of the service worker code.

Here's the IndexedDB db code.

In essence, whenever a call is made for a z/x/y.vector.pbf asset, it is redirected to the database first and then to the network.

For working demo, direct your Android phone to http://zikes.website/ and follow through .. it's a work in progress as i type this, so please forgive that it may not work straight of the bat. I will listen to this thread for questions.

If you find this info useful, and as a form of gratitude, I will ask for a 'like' or 'tweet' from http://zikes.website/ ;)

@lucaswoj : you may remember me (a fellow countryman if i understand correctly) - if offline tiles are the sole motivation for custom source types, you may want to reconsider. All but Safari support service workers as i type this, and service workers arguably solve this problem at a lower layer.

I will be happy to contribute the aforementioned code in some example sandbox of mapbox if you'd like me to.

[pronebird]

Wow thanks @remster for sharing this!

[pronebird]

@remster how do you override fetch? Based on mapbox source they use XHR in utils.js. I don't see how I can intercept XHR requests.

[remster]

service workers are below XHR - you intercept EVERY fetch from all the assets.

self.addEventListener('fetch', function(event) {
event.respondWith(/HERE/);
});

[pronebird]

@remster oh wow thanks for the link. I just realized you were talking about service workers, not web workers.

[pronebird]

@lucaswoj
We are building an app that needs offline support, generally speaking we don't want to have 3rd party traffic whatsoever. Right now it's a compromise to cache as much as we can on client, but in future we'll have to decide in favor of solution that supports offline slice of data (limited to low zoom levels, i.e not much data).

Is business model of Mapbox a constraint in this situation? I don't understand why wouldn't you supply offline maps (or cache) especially when mobile SDKs support them already. There is literally nothing that stops you from doing this.

@remster Thanks again. I found your implementation unnecessary complex for my needs, so to conclude I resolved this with CacheStorage that simply caches all requests to tiles.mapbox.com and api.mapbox.com. The map feels much faster now!

Something like this should totally suffice:

this.addEventListener('install', function (event) {
  console.log('Installing Service Worker');
  event.waitUntil(this.skipWaiting());
});

this.addEventListener('activate', function (event) {
  event.waitUntil(this.clients.claim());
});

this.addEventListener('fetch', function(event) {
  var url = event.request.url;

  if(url.startsWith('https://') && (url.includes('tiles.mapbox.com') || url.includes('api.mapbox.com'))) {
    event.respondWith(
      caches.match(event.request).then(function(resp) {
        return resp || fetch(event.request).then(function(response) {
          var cacheResponse = response.clone();
          caches.open('mapbox').then(function(cache) {
            cache.put(event.request, cacheResponse);
          });
          return response;
        });
      })
    );
  }
});

[MADHUSUDHANRAO527]

Android Mapbox : How to Load tiles from sd card of the mobile[offline]

I am new to mapbox android.I am able to place tiles from json. But How can i load tiles from internal storage/SD card of the mobile.

I am using below code for local storage - which is not working

RasterSource chicagoSource = new RasterSource("chicago-source", new TileSet("tileset", "asset://" + Environment.getExternalStorageDirectory().getAbsolutePath()
+ "/OverlayTiles/{z}/{x}/{y}.png"));
mapboxMap.addSource(chicagoSource);

            RasterLayer chicagoLayer = new RasterLayer("chicago", "chicago-source");
            mapboxMap.addLayer(chicagoLayer);

I am able to load from project assets folder - using below code, it might help someone

RasterSource chicagoSource = new RasterSource("chicago-source", new
TileSet("tileset","asset://OverlayTiles/{z}/{x}/{y}.png"));
mapboxMap.addSource(chicagoSource);
mapboxMap.addSource(chicagoSource);

            RasterLayer chicagoLayer = new RasterLayer("chicago", "chicago-source");
            mapboxMap.addLayer(chicagoLayer);

[allthesignals]

If MapboxGL uses service workers, why don't I see them in the Chrome inspector under the application tab under Service Workers?

I ask because I have a related issue: I need to intercept the fetch request. To do this, I'm using PretenderJS, which should override the default XMLHttpRequest with a mock object. However, the fetch for new vector tiles still isn't being intercepted. I'm not sure why.

One possibility is that the XMLHttpRequest object in the service worker scope isn't being overridden.

That leads me to want to try to listen to the service worker fetch event, however, where do I use this.addEventListener('fetch')? I would like to listen to this event but I don't have direct access to these service workers (as far as I understand).

EDIT:
Thanks @ryanhamley! My mistake. I found a good thread on this issue here: https://stackoverflow.com/questions/53401528/how-to-control-the-xmlhttprequest-object-on-an-html5-web-worker/53402363#53402363.

[ryanhamley]

If MapboxGL uses service workers, why don't I see them in the Chrome inspector under the application tab under Service Workers?

@allthesignals GL JS uses web workers, not service workers so they wouldn't show up in that tab. You can see objects that were downloaded and processed in web workers in the Network tab and you can see activity in the workers if you do a profile in the Memory tab.

Vector tiles are downloaded and parsed in the web workers. I'm not familiar with PretenderJS but presumably it's having trouble accessing the scope of the worker threads. I believe this is something you'll have to clarify with the maintainers of that library and/or ask on StackOverflow.