Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All addon updaters will not work on blender 2.79a and earlier (OSX only, SSL issue) #39

Open
schroef opened this issue Feb 22, 2018 · 36 comments

Comments

@schroef
Copy link
Contributor

schroef commented Feb 22, 2018

IMPORTANT UPDATE (via @TheDuckCow)

Due to a recent change in GitHub's allowed SSL standards, Blender 2.79a and earlier users on Mac OSX will not be able to use the updater, and will see the below screenshot. This only applies to users of Mac OSX blender versions 2.7X and GitHub hosted repositories. There are no issues for GitLab or Bitbucket hosted repositories, and Github-hosted addons continue to work fine on linux & windows. No fix is currently available for OSX blender <2.8 users.

Update February 2019: GitLab is now affected in the exact similar way. Note that Bitbucket-hosted repositories will still function in 2.79 and earlier; all work appropriately in 2.8.

ssl_error_updater
What OSX users would see prior to the v1.0.5 release

Update - release of v1.0.5

The latest release of the updater code does not fix this issue (since it is a server side/blender compiling lack of compatibility), but it does at least more directly acknowledge the error and provide the user with a manual-download solution if affected. Those not affected with continue to see the updater working as normal. Behavior for those affected is as follows:

ALT

ORIGINAL ISSUE POST

(Original issue title: After network downage is get a tlsv1 error)

rv 1.0.4
bl 2.78c
OSX

After my internet went down i see this error
ssl tlsv1_alert_protocol_version

Weird thing is internet works just fine and before that the updater worked just fine. I also see this in the console
Exception in thread Thread-3: Traceback (most recent call last): File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/addon_updater/addon_updater.py", line 1309, in async_check_update self.check_for_update(now=now) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/addon_updater/addon_updater.py", line 1035, in check_for_update self.get_tags() # sets self._tags and self._tag_latest File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/addon_updater/addon_updater.py", line 523, in get_tags self._prefiltered_tag_count = len(all_tags) TypeError: object of type 'NoneType' has no len()

@schroef
Copy link
Contributor Author

schroef commented Feb 22, 2018

It also shows ssl.c:645

Tried googling it, but it to complicated for me

@TheDuckCow
Copy link
Collaborator

TheDuckCow commented Feb 23, 2018 via email

@schroef
Copy link
Contributor Author

schroef commented Feb 24, 2018

I did some more digging and trying to find something. I've got a feeling something changed perhaps in the API method. Because all worked fine just 2 days ago.

When check the get_raw base URL request, this is the part where the warnings are raised. If i check the error outside the try: function i see this in the console. That is the function in addon_updater.py #601

`Traceback (most recent call last):
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 1254, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1106, in request
self._send_request(method, url, body, headers)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1151, in _send_request
self.endheaders(body)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1102, in endheaders
self._send_output(message_body)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1260, in connect
server_hostname=server_hostname)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 377, in wrap_socket
_context=self)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 752, in init
self.do_handshake()
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 988, in do_handshake
self._sslobj.do_handshake()
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 633, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 1344, in async_check_update
self.check_for_update(now=now)
File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 1063, in check_for_update
self.get_tags() # sets self._tags and self._tag_latest
File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 544, in get_tags
all_tags = self._engine.parse_tags(self.get_api(request), self)
File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 636, in get_api
get = self.get_raw(url)
File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 613, in get_raw
result = urllib.request.urlopen(request)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 1297, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 1256, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)>`

@TheDuckCow
Copy link
Collaborator

TheDuckCow commented Feb 26, 2018

Hey there one followup, I see it says 2.78c in some of your paths. Just to give it a spot check, could you try installing and seeing if the same issue occurs in the latest official blender 2.79 branch? Just want to rule out any chance it's a build-specific issue, because after some quick reading it seems potentially related to mismatched SSL versions within the python library, but only a guess so far.

Also one thing I did actually just fix in the latest dev branch here is the line self._prefiltered_tag_count = len(all_tags), so maybe also give it a try downloading the addon updater dev branch

@schroef
Copy link
Contributor Author

schroef commented Feb 28, 2018

@TheDuckCow

Just tried 2.79 and i see same error;

Traceback (most recent call last): File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/addon_updater/addon_updater.py", line 1309, in async_check_update self.check_for_update(now=now) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/addon_updater/addon_updater.py", line 1035, in check_for_update self.get_tags() # sets self._tags and self._tag_latest File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/addon_updater/addon_updater.py", line 523, in get_tags self._prefiltered_tag_count = len(all_tags) TypeError: object of type 'NoneType' has no len()

Wondering what happened, if its either OSX change. But i didnt do any system update nor did i update python. My other guess something with the API of gitHub changed.

All worked fine last week, i tried testing something for @nutti, which also added a issue about nested folders. Suddenly all my versions stopped working. The original addon as well as my adjusted one.

Ill try that dev version and see if that works.

@schroef
Copy link
Contributor Author

schroef commented Feb 28, 2018

Okay that was a quick one, still see a tlsv1 error, but this time it says _ssl.c:720 whereas before it said _ssl.c:645

This is console output on that dev branch
Check update pressed, first getting current status addon_updater_demo updater: Starting background checking thread addon_updater_demo BG thread: Checking for update now in background Checking for update function addon_updater_demo Updater: Read in json settings from file Getting tags from server master branch found, no releases {'zipball_url': 'https://api.github.com/repos/cgcookie/blender-addon-updater/zipball/master', 'name': 'Master'} addon_updater_demo: Wrote out updater json settings to file, with the contents: {'last_check': '2018-02-27 21:19:11.874183', 'just_restored': False, 'ignore': False, 'just_updated': False, 'version_text': {}, 'update_ready': False, 'backup_date': ''} No version strings found text: Master addon_updater_demo: Wrote out updater json settings to file, with the contents: {'last_check': '2018-02-27 21:19:11.874183', 'just_restored': False, 'ignore': False, 'just_updated': False, 'version_text': {}, 'update_ready': False, 'backup_date': ''} addon_updater_demo BG thread: Finished checking for update, doing callback

@TheDuckCow TheDuckCow added the bug label Mar 3, 2018
@TheDuckCow
Copy link
Collaborator

IMPORTANT UPDATE

There appears to have been a change in GitHub's api authentication/certificate handshakes. I have tested multiple different addons using the updater module hosted on GitHub, and they all now fail to work with this same SSL error. I am looking into GitHub's API release documentation to see if there's any official changelog which could be referred to.

More information to come soon as I explore this issue further and come to a solution, but in the meantime: all GitHub hosted addons with updater code will fail to authenticate.

Note that the GitHub & GitLab implementations still function correctly without issue

@schroef Just an FYI I'm editing your initial post accordingly to flag this banner

@TheDuckCow TheDuckCow added Known Issue and removed bug labels Mar 3, 2018
@TheDuckCow
Copy link
Collaborator

TheDuckCow commented Mar 3, 2018

Minor update, further exploring - this issue is likely related to this notice about depreciated cryptographic connection methods.

https://developer.github.com/changes/2018-02-01-weak-crypto-removal-notice/

If this is the case, then unfortunately this is a more fundamental declaration of no support for the level / version of SSL packaged with blender to date so far. Still to be confirmed further.

Supporting evidence: The following set of code works within terminals with more recent SSL versions without issues, but then fails when run within the blender console.

import ssl
import urllib.request

print(ssl.OPENSSL_VERSION)
api_url = 'https://api.github.com'
user = "cgcookie"
repo = "blender-addon-updater"

url = "{}{}{}{}{}{}".format(api_url,"/repos/",user,"/",repo,"/tags")

request = urllib.request.Request(url)
context = ssl._create_unverified_context()
result = urllib.request.urlopen(request,context=context) # issue occurs here within blender
result_string = result.read()
result.close()
get = result_string.decode()
print(get)

@TheDuckCow
Copy link
Collaborator

One point of further encouragement: It appears this code snipped does work in blender 2.8, meaning the version of SSL is updater there moving forward with newer blender versions. I confirm it does NOT work with blender 2.79a (or earlier) on OSX. Will be trying on other operating systems too.

@TheDuckCow TheDuckCow changed the title After network downage is get a tlsv1 error Addon updaters hosted on GitHub not working for blender 2.79a and earlier (minimally OSX) Mar 3, 2018
@schroef
Copy link
Contributor Author

schroef commented Mar 5, 2018

But one thing i dont understand, that ssl or tlvs library is not something which seems to come with blender. It is something used in Python library, right?

On the other this will make this addon not working until blender 2.8 comes out. That will take months i guess

@schroef
Copy link
Contributor Author

schroef commented Mar 8, 2018

How does this thing handle for WIndows users? I mean they are using the same Python library right, why would it work for them, because the python is the same. The SSL is the same for all systems right or do different systems handle different SSL systems. That would be very weird.

@TheDuckCow
Copy link
Collaborator

It would seem that, oddly enough, the OSX build is not using the same SSL library. I personally tested on both windows and Linux and it works fine, and newer builds of OSX (ie blender 2.8) also work fine. I agree very weird, but I have experienced some cases where one build is even missing certain libraries entirely (at one point numpy was not there); I also tested in several versions of blender OSX, seems like it's been consistently and outdated library compared to the others.

@schroef
Copy link
Contributor Author

schroef commented Mar 8, 2018

Okay so that a "mistake" in the builds then, am i right?

What a shame, it was working perfect. Cant this installer work ussing https then?

@TheDuckCow
Copy link
Collaborator

I want to say a "mistake", but it's one that was recurring in previous builds, so not sure how to call that. Regardless yes it is a shame.. though if anything, I would say lucky that at least Windows and Linux are okay, it could have been all three not working.

There could be a workaround which involved packaging a custom build of an ssl library with the addon and have the updater rely on that over blender's built in module, but it would be a lot of extra code and potentially introduce other bugs/oddities that I wouldn't really call it worth the tradeoff.

@schroef
Copy link
Contributor Author

schroef commented Mar 8, 2018

Could i let it check for updates only perhaps or does that also need the SSL. I think it does because the first error popsup when its trying to get to the release headers. Im just wondering why getting a json file needs to be done using a SSL connection. Doesnt make sense to me, im no web guru. But a secure connection for a simple json file seems a bit over the top or am i wrong here?

@TheDuckCow
Copy link
Collaborator

Unfortunately not even checking. The thing is, the API now refuses any connection to the API. I recommend trying the code snipped posted above, try once in a normal "terminal" window outside of blender, and once in an OSX blender build.

As I said, there's a possibility to include an SSL library to use instead and do a relative import of that instead of through the bundled SSL. but... not trivial. Especially since the urllib itself imports ssl, so you'd have to duplicate multiple modules locally on top of providing an external SSL lib. I would be happy to see if it's possible to do within your own repository, partially as a curiosity, but I don't think it's code I'll want to introduce into the master repo here.

@schroef
Copy link
Contributor Author

schroef commented Mar 9, 2018

Have just tried simpel urllib.request.urlopen that is indeed the bugger. Get the same error. I think I just need to pass for now and wait.

I will add a simpel wm.open.url button zo user van check online quickly.

Thanks for the help. Hope my releases adds perhaps come in handy. Once we are at 2.8 I'll give another go, if I don't forget.

Ps are there already builds? 2.79 isn't that old

@TheDuckCow
Copy link
Collaborator

Blender 2.8 is a big transition from the current v2.79, similar to how 2.49 had a big jump to 2.5 (if you were a user back then). In other words, 2.79 will have several releases maybe in parallel to 2.8, while 2.8 is more like a build from the ground up. Adjusted UI, different APIs in some cases, new core features etc. 2.8 is pretty much experimental for now but yes builds do exist, not with all functionality. Learn more here :) https://www.blender.org/2-8/

@schroef
Copy link
Contributor Author

schroef commented Mar 9, 2018

Cool, yes i did try blender before the better GUI. But i was horrified by it. Tried it a couple times but was just not working for me

@schroef
Copy link
Contributor Author

schroef commented Mar 9, 2018 via email

@TheDuckCow TheDuckCow changed the title Addon updaters hosted on GitHub not working for blender 2.79a and earlier (minimally OSX) Addon updaters hosted on GitHub not working for blender 2.79a and earlier (OSX only) Mar 13, 2018
@TheDuckCow
Copy link
Collaborator

Minor planned update, as more and more people are reporting this issue - I will implement a small "patch" which will at least specifically recognize the SSL error and prompt a different, more tailored message (e.g. in place of "check internet"), and provide an option to directly visit the associated website for updating.

@schroef
Copy link
Contributor Author

schroef commented Mar 13, 2018

That would mean for OSX users view a completely different GUI i think. But that way the other users would still profit from the addon.

@TheDuckCow
Copy link
Collaborator

Perhaps but not necessarily; essentially I will set an overwriting flag to trigger the "manual download" popup instead, which would use the link defined by addon_updater.website in the addon_updater_ops.py file.

In this way, it would not actually be hardcoded for OSX users, but hard coded to the specific error; if by chance the issue becomes resolved (unlikely), then it would return to being functional without any further changes.

It's also worth stating, that if updating is a very critical feature for an addon for all* users, the functionality still works fully when the repository is hosted on GitLab or Bitbucket; in an absolute workaround scenario, a second repository could be used exclusively for the purpose of checking against tags. In this scenario, even if the actual code is not hosted on GitLab/Bitbucket, tag names could be still set there in parallel to the GitHub hosted repository (with addon_updater.manual_only being to True). This is not very convenient to maintain by any means, and also would remove the ability to directly update (as opposed to just check for updates) for all users even non OSX.

@jasperges
Copy link

The issue doesn't seem to be Blender specific, but Python 3.5 specific. When I run the snippet posted above with Python 3.5 it gives me the error, with Python 3.6 it's fine.

Not that it helps in any way to resolve the issue, but I checked the different OpenSSL versions on macOS and Linux (Fedora 27):

  • macOS:

    • Python 3.5 and Blender with Python 3.5: OpenSSL 0.9.8zh 14 Jan 2016
    • Python 3.6 and Blender with Python 3.6 (buildbot): OpenSSL 1.0.2n 7 Dec 2017
  • Linux:

    • Python 3.5: OpenSSL 1.1.0g-fips 2 Nov 2017 (built from source with OpenSSL installed with the package manager, that explains the recent OpenSSL version here.)
    • Blender with Python 3.5: OpenSSL 1.0.2j 26 Sep 2016
    • Python 3.6: OpenSSL 1.0.2m-fips 2 Nov 2017
    • Blender with Python 3.6 (buildbot): OpenSSL 1.0.2j 26 Sep 2016

TheDuckCow added a commit that referenced this issue Mar 25, 2018
Addressing this issue thread: #39
@TheDuckCow
Copy link
Collaborator

Just pushed an update (to the dev branch, will be in master soon)which shows this for users encountering the error. The button opens the website as entered via updater.website where they should be able to download and instead in the normal fashion.

ssl_error_open

@schroef
Copy link
Contributor Author

schroef commented Mar 26, 2018

Thats is sweet!, i think thats better for me now. I can get rid of my button i created which does the same, but it does that for all users.

I just need to build in my releases section then, hope that still works.

@schroef
Copy link
Contributor Author

schroef commented Mar 26, 2018

Sweet works nice in 2.78c

I did get an error, there is a missing comma character in thea addon_update.py #567
Missing character
screen shot 2018-03-26 at 12 55 21 pm

Works fine after adding the comma
screen shot 2018-03-26 at 12 54 48 pm

Works fine now for me in 2.78 and 2.79 OSX 10.13.1

@schroef
Copy link
Contributor Author

schroef commented Mar 26, 2018

PS can we ignore this new error in the console then"

Traceback (most recent call last):
  File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/TheaForBlender/addon_updater.py", line 1366, in async_check_update
    self.check_for_update(now=now)
  File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/TheaForBlender/addon_updater.py", line 1103, in check_for_update
    link = self._tags[0]["assets"][0]["browser_download_url"]
KeyError: 'assets'

@TheDuckCow
Copy link
Collaborator

TheDuckCow commented Mar 26, 2018 via email

@schroef
Copy link
Contributor Author

schroef commented Mar 26, 2018

ahh okay, i wanted to test this dev version and see how it working.

PS did i see it correct you added the "releases" options like i added in my fork?

@TheDuckCow
Copy link
Collaborator

TheDuckCow commented Mar 26, 2018 via email

@schroef
Copy link
Contributor Author

schroef commented Apr 28, 2018

I was looking at a different Render Engine addon and i noticed that this has some kind of updater which does work in 2.78c. WHat i dont get is that it also uses the request url but doesnt give an error. It simply reads a txt file. I thought i tried doing a real simple test like this but the i was thinking the URL request simply doesnt work. Perhaps this works cause its using Bitbucket?

urllib.request.urlopen("https://api.bitbucket.org/1.0/repositories/coronablender/render_corona/raw/master/version.txt").read().decode('utf-8').strip()

Perhaps worth a try to test, but the current version is already good of course

@TheDuckCow
Copy link
Collaborator

Yes, this is certainly of course possible - they are just accessing they plain text file to get a file directly from the server (where file could be an html page like this here, or any raw text file). The reason it works is that it's not using the API at all, it's just accessing the file the way a user might via a webbrowser. The other thing is, even if this person were using the addon updater (which uses APIs), they are hosted on bitbucket - as a reminder, the issues above only applies to the mac version of blender and for repositories hosted on github. Any other combo, it would still be working fine.

It would be theoretically possible to build a "scraper" which gets files and versions from github by just iterating over webpages, but that's not a very stable approach (webpage formats may change, for example). But custom implementations like what corona render uses would work fine too, it would just be up to you to define how it works.

@schroef
Copy link
Contributor Author

schroef commented Apr 29, 2018

I thought it was cause of BitBUcket and how that site handles api orso. Im no guru in this so was a long shot.

I tried using simple request on GitHub and that was indeed declined like the other method

@TheDuckCow
Copy link
Collaborator

Ah you are correct, I didn't realize the same rules were also being applied to access to their webpages/over port 80 too. So I guess that is correct, it would not be a functional workaround.

@TheDuckCow TheDuckCow reopened this Jun 5, 2018
@TheDuckCow TheDuckCow changed the title Addon updaters hosted on GitHub not working for blender 2.79a and earlier (OSX only) Addon updaters hosted on GitHub/GitLab not working for blender 2.79a and earlier (OSX only) Mar 3, 2019
@TheDuckCow
Copy link
Collaborator

Updated again to announce that this issue now affects all updating engines, including Bitbucket. However all three engines are fully functional in all versions of 2.8 and current 2.9 alpha.

@TheDuckCow TheDuckCow changed the title Addon updaters hosted on GitHub/GitLab not working for blender 2.79a and earlier (OSX only) All addon updaters will not work on blender 2.79a and earlier (OSX only, SSL issue) May 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants