Rework internals

[twpayne]

This is a work-in-progress PR to update the internals of chezmoi to facilitate features like #423 and #424.

[twpayne]

@zb140 this isn't ready for running on Windows yet, but here's a very quick overview of what this PR hopes to achieve.

The primary goal is to enable new functionality that would be very tricky to implement with chezmoi's internal architecture in its current state. Specifically, these are:

• Implement chezmoi status to show the status of the destination state. This is a three-way comparison between the current destination state, the target state, and the last destination state written by chezmoi. The idea is that if the current destination state is not equal to the last destination state written by chezmoi then the target has been modified by a third party and you can inform the user and take appropriate action. To achieve this, chezmoi will store a hash of the last destination state written in its persistent state and compare this with the equivalent hash of the current destination state.

• Provide more consistent handling of edge cases where multiple features intersect. One example of this is the interaction of chezmoi add, chezmoi apply, .chezmoiignore, and .chezmoiremove. The refactor should make it easier to detect when, say, a user adds a target that matches .chezmoiignore or .chezmoiremove. Another example, is how should encrypted scripts that are also templates be handled by chezmoi edit? There are many such edge cases.

chezmoi's internal architecture at the moment isn't terrible, but it does carry a fair amount of technical debt from its evolution over the past 18 months. Now chezmoi is stabilizing it's time for a clean-up.

Internally, the clean-up does a number of things:

• It makes a clear distinction between the source state, the target state, and the destination state. In v1 there is only the source state and the target state, which makes it hard to hook in the chezmoi status functionality.

• Representations have fewer assumptions built into them. For example, v1 packs the "private" and "executable" attributes into a single POSIX-like mode, and then has to unpack them when an individual attribute is needed. This complicates the Windows implementation as (as your work has been so invaluable in showing) "private" on Windows doesn't mean the same as "private" on POSIX. Now all of this is much cleaner and Windows support should be better.

• Internally, I'm hoping that I can use forward-slash separated paths everywhere, which will simplify a lot of logic when it comes to comparing and matching filenames. I'm not 100% sure that this will work, but I'm crossing my fingers.

• chezmoi v1 initially mutated file systems and then had running scripts and storing state bolted on to that. v2 unifies all of these into a single "system" that is mutated, which makes a lot of functionality much easier to implement. For example, modifying a tar archive "system" becomes writing files to a tar archive and executing a script on a git diff "system" becomes including the script's contents in the diff output.

At the moment, I still have a number of unanswered questions that are not reflected in the code yet, so I wouldn't recommend diving in here unless you're particularly keen or bored :)

[zb140]

This sounds super exciting! Let me know how I can help.

[zb140]

🎉 This is amazing!

[twpayne]

Thanks @zb140 - I'll open an issue explaining how to beta test in the next hour or so.