ICloud integration gives invalid auth w/ app-specific pw

[jkrall]

The problem

I'm attempting to setup the iCloud integration w/ an app-specific password, per the instructions here.

Upon hitting submit, I get a short spinner and then "Invalid authentication" — and I cannot proceed beyond this error to finish setting up the icloud integration. I've also tried the suggested troubleshooting step of removing the .storage/icloud directory, to no avail. (which is created upon attempting to setup the integration, but removing it has no affect on the issue)

I have confirmed that I've entered the app-specific password correct, and attempted several different passwords I've generated on appleid.apple.com.

What is version of Home Assistant Core has the issue?

core-2021.7.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

icloud

Link to integration documentation on our website

https://www.home-assistant.io/integrations/icloud/#app-specific-passwords

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2021-08-03 23:05:32 ERROR (SyncWorker_40) [pyicloud.base] Missing apple_id field
2021-08-03 23:05:32 ERROR (MainThread) [homeassistant.components.icloud.config_flow] Error logging into iCloud service: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field'))

Additional information

No response

[probot-home-assistant]

icloud documentation
icloud source
_{^{(message by IssueLinks)}}

[probot-home-assistant]

Hey there @Quentame, @nzapponi, mind taking a look at this issue as it has been labeled with an integration (icloud) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[Salvora]

same here, app password doesn't work.

[N3rdix]

same for me

[MattLParker]

same: these occur in logs
Error logging into iCloud service: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field'))

[shivindera]

Bump!

[shailyglobal83]

i am facing the same issue.

Any update on same

[shailyglobal83]

i am waiting for this issue to get resolve ASAP because my presence detection automation is completely blocked because of that issue .Please respond if there is any update regarding this issue. Thanks in advance.

[tommekevda]

same here

[pauly7300]

I'm experiencing the same running 2021.8.8. Both with my actual icloud pw as well as app specific pw. I've also tried with 2 separate icloud accounts. Additionally i've spun up multiple VM's to try and isolate all other factors and have even downgraded Core to 2021.7.x which is where I had it running last but even that doesn't seem to work. My assumption was the the version of Core was potentially the issue but now I'm wondering if the integration itself has a 'version' that isn't manageable within the context of HA itself and potentially that is the issue. I've spent quite a bit of time trying to isolate this and can't figure out what is different...

The specific behavior for me is after entering my un/pw on the below screen, I get the Apple prompt immediately asking to allow a login. I proceed to the 6 digit code dialog but the dialog below just spins continuously for a very long time and i never get prompted to enter the code.

[david-schwartz-by]

I have the same experience as @pauly7300

I'm experiencing the same running 2021.8.8. Both with my actual icloud pw as well as app specific pw. I've also tried with 2 separate icloud accounts. Additionally i've spun up multiple VM's to try and isolate all other factors and have even downgraded Core to 2021.7.x which is where I had it running last but even that doesn't seem to work. My assumption was the the version of Core was potentially the issue but now I'm wondering if the integration itself has a 'version' that isn't manageable within the context of HA itself and potentially that is the issue. I've spent quite a bit of time trying to isolate this and can't figure out what is different...

The specific behavior for me is after entering my un/pw on the below screen, I get the Apple prompt immediately asking to allow a login. I proceed to the 6 digit code dialog but the dialog below just spins continuously for a very long time and i never get prompted to enter the code.

<img alt="2021-08-30_13-34-07" width="468" src="https://user-images.githubusercontent.com/11494052/131388098-1df4ba36-513f-4ce4-8cff-b9648103beff.png"

[sarabveer]

Same issue on 2021.9.1

Error in logs:

Error logging into iCloud service: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field'))

[thomasfr]

Same issue, same error, same logs here on latest HAOS with latest Core, Supervisor versions. Can I help somehow to fix it?

[dsfaller]

Same issue here with app-specific password... I played a little bit with the underlying code samples of pyiCloud (https://github.com/picklepete/pyicloud) and the sample code showed the same error when using my AppleID / app-specific password... raising a ticket there and hoping that the developer will react.

[thomasfr]

Thanks for the update

[ushroff]

I'm experiencing the same running 2021.8.8. email address and password are accepted. But verification code is accepted and constantly rejected by the iCloud integration app.
Logger: homeassistant.components.icloud.config_flow
Source: components/icloud/config_flow.py:272
Integration: Apple iCloud (documentation, issues)
First occurred: 2:11:28 AM (2 occurrences)
Last logged: 2:11:37 AM

Failed to verify verification code: (401)

[mada199122]

I have the same issue too...

[Andavin]

Is there going to be a resolution for this? It seems as though this hasn't been noticed by a developer

[dsfaller]

I did more digging with the underlying pyicloud library (see comment in my issue there) - seems like pyicloud is emulating to be a web browser and not using any official API. Apparently, Apple changed the login flow or rejects app-specific passwords completely now - in any case the library no longer receives the necessary login tokens to work.
My conclusion is that we need a rewrite of pyicloud for app-specific passwords or another library for the icloud plugin in Home Assistant... :(

[mada199122]

bad news..
thanks any way....

[github-actions]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[rasmusbe]

still an issue

[wolph]

I just report those mails as spam since I can't unsubscribe from them. After a little while gmail learns to trash them :)

Please set up a filter instead! This could cause false-positives for other people and may mean someone misses an indicator of an account breach.

Any mail that you cannot unsubscribe from is spam in my book. Perhaps this way Apple will fix their broken system at some point

[LewisSpring]

I just report those mails as spam since I can't unsubscribe from them. After a little while gmail learns to trash them :)

Please set up a filter instead! This could cause false-positives for other people and may mean someone misses an indicator of an account breach.

Any mail that you cannot unsubscribe from is spam in my book. Perhaps this way Apple will fix their broken system at some point

I don't disagree!

[polskikrol]

Here is a link to the feature request in v3: gcobb321/icloud3_v3#78. Doest appear that either the development branch nor the integrated feature in core will support app specific passwords anytime soon. Everything runs on an older python implementation (https://pypi.org/project/pyicloud/) which essentially recreates the web calls rather than using any modern API which Apple now supports. The lack of app specific passwords is also an issue in this python code as well (picklepete/pyicloud#349). This is truly a double whammy of crud from a security perspective:

• One doesnt want to use / encode / hard code / whatever their master icloud password in some config file on an automation server. App passwords are meant to over come this, where you have a different password for a specific application and can easily revoke or change, without affecting your other logins.
• One doesnt want email spam from the python cloud re-auth occurring a few times daily as tokens age out, which are then ignored, and may result in a true compromise or attack being missed.

In summary, doesnt look good.

[issue-triage-workflows]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[shailyglobal83]

Hi,

Thanks for the response, i Checked it on the latest version and it seems that this issue is not resolved by the team yet. So kindly do not close and it take it on high priority because no one would like to use actual password in configuration So we should have the option to login with app specific password.

[polskikrol]

Agreed. This is not fixed. Many other modules support API based authentication using app specific passwords, but this one does not at the moment. Given many people have iPhones, not sure why this is not prioritized.

[wolph]

Agreed. This is not fixed. Many other modules support API based authentication using app specific passwords, but this one does not at the moment. Given many people have iPhones, not sure why this is not prioritized.

It's not fixed because it can't be fixed. The available apple api's don't expose this information so we can't use app specific passwords for this.

This issue really should be closed and marked as unfixable for the time being

[issue-triage-workflows]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[magicus]

@balloob @frenck Sorry for the ping, but I don't know how to get visibility for this otherwise. The icloud integration is basically abandoned and broken. It needs either to have an active developer assigned as owner, or it should be removed from the core distribution.

It is listed as owned by @Quentame and @nzapponi, but neither of them have responded to any of the open bugs. @nzapponi said in #101816 (comment) that he only contributed a few changes 3 years ago, and is not maintaining the component.

There are about a dozen open bugs on the icloud integration, all of them basically boiling down to the fact that authentication with Apple does not work properly anymore. The failure modes are a bit different, but typically you can install the integration and it works for a while, but then the login with Apple expires, and the problems begin.

[issue-triage-workflows]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[magicus]

Hi bot, this is in no way resolved. The lack of activity is due to the lack of anyone responsible for fixing it. :-/

[issue-triage-workflows]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[magicus]

Hi bot, this is in no way resolved. The lack of activity is due to the lack of anyone responsible for fixing it. :-/

[caseyjmorton]

OK. I'm done waiting on someone else more qualified to fix this. I will jump in and see what I can to to fix it myself. For those of you familiar with Major League, just call me Pedro Serrano. I will start working on PRs for all of this. If anyone has any tribal knowledge on this that they can share to help me get started in the right direction, it would be much appreciated. I cant promise a quick or complete solution, but apparently I'm all we got :)

[wolph]

OK. I'm done waiting on someone else more qualified to fix this. I will jump in and see what I can to to fix it myself. For those of you familiar with Major League, just call me Pedro Serrano. I will start working on PRs for all of this. If anyone has any tribal knowledge on this that they can share to help me get started in the right direction, it would be much appreciated. I cant promise a quick or complete solution, but apparently I'm all we got :)

I've got no clue who major League is but I wish you a lot of luck!

The issue is unfortunately a very difficult one to solve. Apple has no public API available so your only option to do this without the emails is using (or emulating) an apple device which is non-trivial to say the least. Apple really doesn't want 3rd parties to use their private APIs so it's really hard to get that path working.

As I see it you have a few options:

1. Having a dedicated Apple device which is remote controlled to fetch this information
2. Having a virtual apple device which is remote controlled. Not sure what the current state is but I know there was a time that a homebrewed apple device could access the APIs
3. Reverse engineer the Apple authentication to spoof having a real apple device. This one is probably prohibitively difficult to achieve, but if the homebrew option works than you could try this method.

[vajonam]

There is already working HACS icloud integration that works great. https://github.com/gcobb321/icloud3. This integration is dead I won't wast any time on this. Just my $0.02.

[caseyjmorton]

Understood. Having looked at the HACS one, consider my previous rant retracted.

[caseyjmorton]

@wolph It looks like someone else has done the real legwork and gone far beyond in a separate project. I guess, I'm just going to migrate to that one as I suggest everyone else who comes across this thread do as well.

I've got no clue who Major League is but I wish you a lot of luck!

Its a 1980's movie about American baseball. I'm honestly not sure how well the humor translates, but IMHO one of the funniest movies of all time. The exact line referenced is "I say F you, Jobu! I'll do it myself!" Its definitely worth a stream.

[magicus]

With that said, the icloud3 integration is very "heavy" and comes with a GUI that does not fit with Home Assistant, in contrast to this integration. If it would work, I would switch back from icloud3 right away.

Basically all the fixes that makes icloud3 work resides in changes to the pyicloud library, which this integration also uses. Unfortunately the original author of pyicloud has virtually abandoned the project, so it has not been updated upstream for a long time, but instead several patched version has appeared in different projects.

I started looking at what changes icloud3 had done to the pyicloud library. Unfortunately, the author of icloud3 have made some drastic refactorings that make it really hard to track what is going on. But I'm trying, from time to time, to spend a bit of time getting closer to untangle that mess, to see which changes are really made. I am pretty confident that some of these changes holds the key to getting this integration working again.

Ideally, pyicloud would get a new maintainer and all these patches could be upstreamed to it.

[vajonam]

@wolph It looks like someone else has done the real legwork and gone far beyond in a separate project. I guess, I'm just going to migrate to that one as I suggest everyone else who comes across this thread do as well.

Just follow the docs, and you will be good. with v3 its been made a lot simpler to setup.