403 (Forbidden) FCM Errors

[cbratschi]

NOTE: Please test in a least two browsers (i.e. Chrome and Firefox). This
helps with diagnosing problems quicker.

Setup

Please provide the following details, the more info you can provide the
better.

Operating System: Linux
Node Version: 11.x
web-push Version: 3.3.4

Please select any browsers that you are experiencing problems with:

• Chrome
• Firefox
• Opera for Android
• Samsung Internet Browser
• Other

Please list the browsers you are have tested this, including the version
of the browser (i.e. Chrome Beta, Firefox Beta etc).

Problem

Please explain what behaviour you are seeing.
Some of our messages sent to the FCM endpoint (https://fcm.googleapis.com/fcm/send/*) fail with 403 (Forbidden). Because we are not removing such failed token from our database the amount keeps increasing. Currently about 2% of all our web push tokens are affected, more than 1000 tokens. These are all VAPID tokens.

Does anyone else have similar issues? Do you discard such failing tokens?

Is this documented anywhere by Google?

Expected

Please explain what you expected to happen

Features Used

• VAPID Support
• GCM API Key
• Sending with Payload

Example / Reproduce Case

Please provide a code sample that reproduces the issue. If there is a
repository that reproduces the issue please put the link here.

Other

Please put any remaining notes here.

[marco-c]

I haven't heard of this problem before. I think you'll have to file a bug for FCM.

[rohit-gohri]

Is this documented anywhere by Google?

https://firebase.google.com/docs/reference/fcm/rest/v1/ErrorCode

[cbratschi]

I informed Google but they could not analyze my failing tokens. It seems all affected tokens are from April to June 2019. To be confirmed soon.

[cbratschi]

Yes, only new tokens are affected.

[cbratschi]

pywebpush made some VAPID exp adjustments: web-push-libs/pywebpush#100

They had the same problem.

[marco-c]

@cbratschi @renegadeHomie can you share the code you are using to send the push notifications with web-push? I can check if it could be the same problem as pywebpush, but I doubt it (as far as I can see, we regenerate the expiration on every call to sendNotification).

[cbratschi]

Hi @marco-c,

We are using an Express service i.e. the module is loaded once and stays in memory. About 10 web pushes are sent a day to about 50k subscribers. We are using 4 child processes to send the pushes as fast as possible.

Here is the central part of the push code:

    const payload = typeof opts.payload === 'string' ? opts.payload:JSON.stringify(opts.payload);
    const optionsVapid = {
        //VAPID
        vapidDetails: {
            subject: opts.vapid.subject,
            publicKey: opts.vapid.publicKey,
            privateKey: opts.vapid.privateKey
        },

        //GCM (or FCM without VAPID)
        gcmAPIKey: opts.gcm,

        //others
        TTL: 0
    };
    const optionsNoVapid = {
        vapidDetails: null,
        gcmAPIKey: optionsVapid.gcmAPIKey,
        TTL: optionsVapid.TTL
    };

The non-VAPID version is used for old GCM tokens migrated to FCM.

Push loop where options points to the array above (array instance is static):

        const subscription = {
            endpoint,
            keys: {
                p256dh: item[2],
                auth: item[3]
            }
        };

        const push = webpush.sendNotification(subscription, payload, options);

        push.then(() => {
           ...
        }, err => {
          ...
        });

Hope this helps.

[marco-c]

I'm assuming the failing ones are all VAPID ones like you said in the first comment, correct?

Could you try running

web-push/src/vapid-helper.js

Line 128 in c8bb8b4

function getFutureExpirationTimestamp(numSeconds) {

and seeing if the expiration is correct? Is your system clock set wrongly by any chance?

[cbratschi]

Yes, these are all VAPID ones, since April 2019. Older tokens seem to work fine. But not all new tokens are affected.

Will check the server time now.

[cbratschi]

The server time is fine.

[cbratschi]

The Java library has issues too: web-push-libs/webpush-java#57

In my opinion this is a Firebase bug but Firebase support does not want to investigate yet.

[wolfgang42]

For whatever it's worth, I was getting this same 403 Forbidden error and it turned out to be because I was decoding my VAPID keys the wrong way on the client (the library I used didn't understand base64url) so they didn't match the ones on the server. I doubt that this is what's causing your problem but this issue was the top search hit for the error message so hopefully I can help someone else.

[dseljan]

I can confirm sending push notifications doesn't work for FCM using VAPID. Getting 403 each time. I'm on latest Chrome version.
I've just tried to send a notification using subscription in Mozilla, everything works as expected.

EDIT: I have managed to send a notification to Chrome! It turns out, if you send an empty payload, you get a status forbidden for some reason.

[berberon]

had the same problem, needed to add "https://" before fcm.googleapis.com/fcm/send and it solved the problem

[cbratschi]

had the same problem, needed to add "https://" before fcm.googleapis.com/fcm/send and it solved the problem

In our cases the URLs are all HTTPS but fail with 403. Currently 2k of about 74k pushes are failing. The number is now more or less stable. We plan to remove those 2k tokens as expired ones soon. There is no other solution right now.

[aranoe]

I have the same problem, getting 403 Forbidden. I switched back to an older one and it works again.

[marco-c]

I wonder if this is the same as #532.

[zhelingwang]

I met this problem . I think the reason why you get a 403 Forbidden is that cache file used by serviceWorker does not update in time.

[marco-c]

Are you still seeing this issue?

[cbratschi]

We consistently get some 403 errors. These values are from our latest sent batch of push notifications:

Host errors:

• 401:wns2-par02p.notify.windows.com: 1
• 403:fcm.googleapis.com: 8
• 500:fcm.googleapis.com: 30

Format: ::