Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update user agent strings for agent aliases #612

Merged
merged 3 commits into from
Apr 15, 2023
Merged

Update user agent strings for agent aliases #612

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
78791c5
add example to fetch latest user agents
takatea Apr 14, 2023
96d21aa
update user agent strings for agent aliases
takatea Apr 14, 2023
18b7efd
doc: update docstring for AGENT_ALIASES and CHANGELOG
flavorjones Apr 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Mechanize CHANGELOG

## next / unreleased

* Updated User-Agent strings to represent modern browser versions. (#612) Thank you, @takatea!


## 2.9.0 / 2023-04-07

### Requirements
Expand Down
91 changes: 91 additions & 0 deletions examples/latest_user_agents.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
require 'mechanize'

class LatestUAFetcher
attr_reader :user_agents

BASE_URL = 'https://www.whatismybrowser.com/guides/the-latest-user-agent'

def initialize
@agent = Mechanize.new.tap { |a| a.user_agent_alias = 'Mac Firefox' }
@user_agents = {}
end

def run
sleep_time = 1

puts 'get chrome UA...'
chrome
puts "sleeping... (#{sleep_time}s)"
sleep 1

puts 'get firefox UA...'
firefox
puts "sleeping... (#{sleep_time}s)"
sleep 1

puts 'get safari UA...'
safari
puts "sleeping... (#{sleep_time}s)"
sleep 1

puts 'get edge UA...'
edge
end

private

def edge
page = @agent.get("#{BASE_URL}/edge")

windows_dom = page.css("h2:contains('Latest Edge on Windows User Agents')")
@user_agents[:edge] = {
windows: windows_dom.css('+ .listing-of-useragents .code').first.text
}
end

def firefox
page = @agent.get("#{BASE_URL}/firefox")

desktop_dom = page.css("h2:contains('Latest Firefox on Desktop User Agents')")
table_dom = desktop_dom.css('+ .listing-of-useragents')

@user_agents[:firefox] = {
windows: table_dom.css('td:contains("Windows")').css('+ td .code').text,
macOS: table_dom.css('td:contains("Macos")').css('+ td .code').text,
linux: table_dom.css('td:contains("Linux")').css("+ td .code:contains('Ubuntu; Linux x86_64')").text
}
end

def safari
page = @agent.get("#{BASE_URL}/safari")

macos_dom = page.css("h2:contains('Latest Safari on macOS User Agents')")
ios_dom = page.css("h2:contains('Latest Safari on iOS User Agents')")

@user_agents[:safari] = {
mac_os: macos_dom.css('+ .listing-of-useragents .code').first.text,
iphone: ios_dom.css('+ .listing-of-useragents').css("tr:contains('Iphone') .code").text,
ipad: ios_dom.css('+ .listing-of-useragents').css("tr:contains('Ipad') .code").text
}
end

def chrome
page = @agent.get("#{BASE_URL}/chrome")

windows_dom = page.css("h2:contains('Latest Chrome on Windows 10 User Agents')")
linux_dom = page.css("h2:contains('Latest Chrome on Linux User Agents')")
macos_dom = page.css("h2:contains('Latest Chrome on macOS User Agents')")
android_dom = page.css("h2:contains('Latest Chrome on Android User Agents')")

@user_agents[:chrome] = {
windows: windows_dom.css('+ .listing-of-useragents .code').first.text,
linux: linux_dom.css('+ .listing-of-useragents .code').first.text,
mac_os: macos_dom.css('+ .listing-of-useragents .code').first.text,
android: android_dom.css('+ .listing-of-useragents .code').first.text
}
end
end

agent = LatestUAFetcher.new
agent.run
p agent.user_agents
79 changes: 49 additions & 30 deletions lib/mechanize.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,54 +87,73 @@ class Error < RuntimeError
# description in parenthesis is for informative purposes and is not part of
# the alias name.
#
# * Linux Firefox (43.0 on Ubuntu Linux)
# * Linux Konqueror (3)
# * Linux Mozilla
# * Mac Firefox (43.0)
# * Mac Mozilla
# * Mac Safari (9.0 on OS X 10.11.2)
# * Mac Safari 4
# * Mechanize (default)
# * Windows IE 6
# * Windows IE 7
# * Windows IE 8
# * Windows IE 9
# * Windows IE 10 (Windows 8 64bit)
# * Windows IE 11 (Windows 8.1 64bit)
# * Windows Edge
# * Windows Mozilla
# * Windows Firefox (43.0)
# * iPhone (iOS 9.1)
# * iPad (iOS 9.1)
# * Android (5.1.1)
# The default User-Agent alias:
#
# * "Mechanize"
#
# Linux User-Agent aliases:
#
# * "Linux Firefox"
# * "Linux Konqueror"
# * "Linux Mozilla"
#
# Mac User-Agent aliases:
#
# * "Mac Firefox"
# * "Mac Mozilla"
# * "Mac Safari 4"
# * "Mac Safari"
#
# Windows User-Agent aliases:
#
# * "Windows Edge"
# * "Windows Firefox"
# * "Windows IE 6"
# * "Windows IE 7"
# * "Windows IE 8"
# * "Windows IE 9"
# * "Windows IE 10"
# * "Windows IE 11"
# * "Windows Mozilla"
#
# Mobile User-Agent aliases:
#
# * "Android"
# * "iPad"
# * "iPhone"
#
# Example:
#
# agent = Mechanize.new
# agent.user_agent_alias = 'Mac Safari'

#
AGENT_ALIASES = {
# TODO: use output from examples/latest_user_agents.rb as the underling data structure
'Mechanize' => "Mechanize/#{VERSION} Ruby/#{ruby_version} (http://github.com/sparklemotion/mechanize/)",
'Linux Firefox' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0',

'Linux Firefox' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:112.0) Gecko/20100101 Firefox/112.0',
'Linux Konqueror' => 'Mozilla/5.0 (compatible; Konqueror/3; Linux)',
'Linux Mozilla' => 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624',
'Mac Firefox' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:43.0) Gecko/20100101 Firefox/43.0',

'Mac Firefox' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 13.3; rv:112.0) Gecko/20100101 Firefox/112.0',
'Mac Mozilla' => 'Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4a) Gecko/20030401',
'Mac Safari 4' => 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; de-at) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10',
'Mac Safari' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/601.3.9 (KHTML, like Gecko) Version/9.0.2 Safari/601.3.9',
'Windows Chrome' => 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.125 Safari/537.36',
'Mac Safari' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 13_3_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Safari/605.1.15',

'Windows Chrome' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36',
'Windows Edge' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36 Edg/112.0.1722.46',
'Windows Firefox' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) Gecko/20100101 Firefox/112.0',
'Windows IE 6' => 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)',
'Windows IE 7' => 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)',
'Windows IE 8' => 'Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)',
'Windows IE 9' => 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)',
'Windows IE 10' => 'Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)',
'Windows IE 11' => 'Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko',
'Windows Edge' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.10586',
'Windows Mozilla' => 'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4b) Gecko/20030516 Mozilla Firebird/0.6',
'Windows Firefox' => 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0',
'iPhone' => 'Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B5110e Safari/601.1',
'iPad' => 'Mozilla/5.0 (iPad; CPU OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1',
'Android' => 'Mozilla/5.0 (Linux; Android 5.1.1; Nexus 7 Build/LMY47V) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.76 Safari/537.36',

'Android' => 'Mozilla/5.0 (Linux; Android 10) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.5615.48 Mobile Safari/537.36',
'iPad' => 'Mozilla/5.0 (iPad; CPU OS 16_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Mobile/15E148 Safari/604.1',
'iPhone' => 'Mozilla/5.0 (iPhone; CPU iPhone OS 16_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Mobile/15E148 Safari/604.1',
}

AGENT_ALIASES.default_proc = proc { |hash, key|
Expand Down