extract links option partially working for the initial target URL

[epi052]

@epi052 Hey I was also looking at this machine and noticed that the extract links option is not working for the initial target URL.
I thought it was just because the filtered was not captured <link hrefs= but adding that in, did not do the job. Example below:
<link href="/js/app.js" rel="preload" as="script"><link href="/js/chunk-vendors.js" rel="preload" as="script">

Add to line 215 in src/extractor/container.rs:

self.extract_links_by_attr(resp_url, links, html, "link", "href");

I was analyzing the code and looks like the robots and directory heuristic are the only ones run on the initial target URL and then dumped into looping with words for the input wordlist. Not sure if its because this was matching the wildcard test. Let me know what you think.

I can open up a new issue for this if more appropriate.

Originally posted by @godylockz in #689 (comment)

[epi052]

@godylockz

looked at this today, yes, the issue was that the initial response was being filtered due to the wildcard response.

if you add the code below to src/scanner/requester.rs:446

log::error!("extracting: {:?}", ferox_response.url().as_str());

and run the scan with -e --dont-filter, you'll see the initial url trigger the extraction logic.

[epi052]

added extraction tho 😁 closing this one, thanks!

self.extract_links_by_attr(resp_url, links, html, "link", "href");

[godylockz]

Thanks boss 👍