Sites Giving 404 error or no profile #1189
Comments
|
@SethFalco I think you were the one who added Anilist, could you take a look at that site? |
|
I added the Anilist, Coil, and RuneScape, I can look at the 3 of them either this evening (UTC + 2) or on the weekend. |
|
Yep, I can confirm that |
|
Hmm, with how Sherlock currently works, I'm wondering if it's possible to cover 2 of them properly. For now, we can move Anilist and Coil to removed sites, maybe? In future it'd be nice to discuss if we can add more errorTypes or options for them to cover these cases? AnilistSo Anilist is failing because the
This could be possible to fix if there was an option to do a custom HTTP request. Like a detection that queries an API or GraphQL and checks if the response matches the success/failure schema. CoilThese are failing because we're not executing JavaScript (which is fair enough) but the message specified only appear after JavaScript is fetched and executed. Sherlock only looks at the initial response, which in the cases of Coil and RuneScape only include the common HTML shared with every page and links to their JavaScript bundles. Coil could be fixed similarly to Anilist. i.e. Make a POST request and checking if the response matches a success/failure schema. The (formatted) initial response for Coil, for example: <!doctype html>
<html lang="en" dir="ltr">
<head>
<meta charset="utf-8">
<meta name="viewport"
content="user-scalable=0,initial-scale=1,minimum-scale=1,width=device-width,height=device-height">
<meta name="theme-color" content="#000000">
<title>Coil</title>
<link rel="icon" type="image/png"
href="https://cdn.coil.com/assets/favicon.png" />
<link href="https://cdn.coil.com/assets/fonts/circular-std/circularstd.css"
rel="stylesheet">
<link href="https://cdn.coil.com/assets/fonts/roboto-mono/roboto-mono.css"
rel="stylesheet">
<link
href="https://cdn.coil.com/assets/fonts/material-icons/material-icons.css"
rel="stylesheet">
<script>document.monetization ? document.monetizationExtensionInstalled = !0 : (document.monetization = document.createElement("div"), document.monetization.state = "stopped")</script>
<script src="https://cdn.coil.com/coil-oauth-wm.v6.beta.js"
defer="defer"></script><noscript id="jss-insertion-point"></noscript>
<link href="/static/css/2.6673ab0d.chunk.css" rel="stylesheet">
</head>
<body>
<div id="root"></div>
<script>!function (c) { function e(e) { for (var r, t, n = e[0], o = e[1], u = e[2], i = 0, a = []; i < n.length; i++)t = n[i], f[t] && a.push(f[t][0]), f[t] = 0; for (r in o) Object.prototype.hasOwnProperty.call(o, r) && (c[r] = o[r]); for (d && d(e); a.length;)a.shift()(); return p.push.apply(p, u || []), l() } function l() { for (var e, r = 0; r < p.length; r++) { for (var t = p[r], n = !0, o = 1; o < t.length; o++) { var u = t[o]; 0 !== f[u] && (n = !1) } n && (p.splice(r--, 1), e = s(s.s = t[0])) } return e } var t = {}, f = { 1: 0 }, p = []; function s(e) { if (t[e]) return t[e].exports; var r = t[e] = { i: e, l: !1, exports: {} }; return c[e].call(r.exports, r, r.exports, s), r.l = !0, r.exports } s.e = function (u) { var e = [], t = f[u]; if (0 !== t) if (t) e.push(t[2]); else { var r = new Promise(function (e, r) { t = f[u] = [e, r] }); e.push(t[2] = r); var n, i = document.createElement("script"); i.charset = "utf-8", i.timeout = 120, s.nc && i.setAttribute("nonce", s.nc), i.src = s.p + "static/js/" + ({}[u] || u) + "." + { 3: "3b00d1c9", 4: "26e58311" }[u] + ".chunk.js", n = function (e) { i.onerror = i.onload = null, clearTimeout(a); var r = f[u]; if (0 !== r) { if (r) { var t = e && ("load" === e.type ? "missing" : e.type), n = e && e.target && e.target.src, o = new Error("Loading chunk " + u + " failed.\n(" + t + ": " + n + ")"); o.type = t, o.request = n, r[1](o) } f[u] = void 0 } }; var a = setTimeout(function () { n({ type: "timeout", target: i }) }, 12e4); i.onerror = i.onload = n, document.head.appendChild(i) } return Promise.all(e) }, s.m = c, s.c = t, s.d = function (e, r, t) { s.o(e, r) || Object.defineProperty(e, r, { enumerable: !0, get: t }) }, s.r = function (e) { "undefined" != typeof Symbol && Symbol.toStringTag && Object.defineProperty(e, Symbol.toStringTag, { value: "Module" }), Object.defineProperty(e, "__esModule", { value: !0 }) }, s.t = function (r, e) { if (1 & e && (r = s(r)), 8 & e) return r; if (4 & e && "object" == typeof r && r && r.__esModule) return r; var t = Object.create(null); if (s.r(t), Object.defineProperty(t, "default", { enumerable: !0, value: r }), 2 & e && "string" != typeof r) for (var n in r) s.d(t, n, function (e) { return r[e] }.bind(null, n)); return t }, s.n = function (e) { var r = e && e.__esModule ? function () { return e.default } : function () { return e }; return s.d(r, "a", r), r }, s.o = function (e, r) { return Object.prototype.hasOwnProperty.call(e, r) }, s.p = "/", s.oe = function (e) { throw console.error(e), e }; var r = window.webpackJsonp = window.webpackJsonp || [], n = r.push.bind(r); r.push = e, r = r.slice(); for (var o = 0; o < r.length; o++)e(r[o]); var d = n; l() }([])</script>
<script src="/static/js/2.0633994c.chunk.js"></script>
<script src="/static/js/main.2dfdce89.chunk.js"></script>
</body>
</html>
RuneScapeHad the same issue as Coil, but I can update this one to use a restful endpoint that only requires query parameters. |
|
I've also checked the other 2 while I was at it. TrackmaniaLadderLooks like the issue for this one is rate limited related? If you try to query for a profile too quickly from the previous, it has a different error message. (From my testing, around 4 times a minute will trigger this.)
babyblogRUSeems they've changed the URL for this one, but they've also made it so not found users result in a redirect which means it can move away from |
We have #861 but it is a little broken at the moment. If that gets fixed I am certain we are able to bring back a lot of sites. This is because many sites check the username availability through a post request. |
|
Nice, the only thing left immediately for this should be TrackmaniaLadder. What's the best move there? Then in the case of an internal server error, unable to connect, or rate limit, it can return a neutral response instead? |
|
It can be removed for now and we can take a further look at such responses later |
|
Closing this in favor of: |
Checklist
Description
There are some sites which comes in result of matched Usernames but tends to give No Profile Page or a 404 Error,
those sites are below..
[+] Anilist: https://anilist.co/user/
[+] Coil: https://coil.com/u/
[+] RuneScape: https://apps.runescape.com/runemetrics/app/overview/player/
[+] TrackmaniaLadder: http://en.tm-ladder.com/_rech.php
[+] babyblogRU: https://www.babyblog.ru/user/info
The text was updated successfully, but these errors were encountered: