[WiP] implications

[michielbdejong]

No description provided.

[michielbdejong]

The whole lookalike-group logic is pretty complex.
A simpler solution to the problem of indistinguishable lines in statements would be to give them an index.
So we add a column in the parsed entry object that is usually 0, but when parsing a bank statement or a timesheet, if two lines are identical (same date, same amount, same description, but still two distinct entries) then we set it to an incremental number, so we can distinguish "the i-th entry on that date that looks like that".

[michielbdejong]

We can call this variable subIndex, and we also add the deleted column for tombstoning. So then a movement implied by a statement has:

• fromComponent, toComponent
• amount, unit,
• timestamp
• subIndex, deleted

The implication links it to at least one statement, with relation 'inner', 'outer', 'delivery', 'consumption', 'production'.

The statement refers to an entry in the database of a neighbouring system or organisation, as evidenced by appearance in a business document, statement, API request body, API response body, or UI gesture.

As such it should have a system identifier and a remote ID that is scoped to that system.

It might be good to also store the documents as evidence?

• contents
• language
• origin
• [grounding is provided by the statements that link it to the movements]
• speech act: API-request, API-response, UI-gesture, CSV-download, business-document

[michielbdejong]

So then import-bank-statement is a sysadmin's command that pipes a CSV file into the documents table
From there it will deterministically generate statements, implications, movements.

[michielbdejong]

How can the user manage the documents that PreJournal holds?
For now, just reset the db and then run commands that trigger the addDocument function.

[michielbdejong]

After adding a document you could check if the statements table is correct given the current set of documents, or just generate the statements for the currently added document. Both trigger paths should eventually be possible.

[michielbdejong]

Maybe the statements table should have a freeform JSON attributes column, since there can be so much diversity in that, and you don't want to lose the information either.

[michielbdejong]

Should we be tracking which document line links to which statement? Then maybe the document should be split into lines while parsing?

[michielbdejong]

A document is batch of messages.

document -> message -> statement -> implication -> movement

[michielbdejong]

A document like a bank statement or a timesheet is actually not just a batch of entries because it also claims to be exhaustive for the dates it covers.

[michielbdejong]

Maybe 'interaction' is a better word for when the state of a PreJournal instance changes due to an interaction with another system. And maybe 'line' is a better word for 'message'.

interaction -> line -> statement -> implication -> movement

[michielbdejong]

The important thing now is that the to and from component are created/chosen correctly and that the attributes go on the statement.

[michielbdejong]

bank statement parser function produce:

• date -> movement timestamp
• comment -> should go onto the statement somehow?
• from -> outer movement fromComponent
• to -> outer movement toComponent
• amount -> movement amount
• balanceAfter -> for sanity checking
• insideFrom -> inner movement fromComponent
• insideTo -> inner movement toComponent
• lineNum -> for linking to a line in the doc
• remoteId (ASN only, ING doesn't have this)

timesheet parser functions produce:

• worker -> movement fromComponent
• project (client : project) -> movement toComponent; see Documentation clarification of "project" and "worker" concepts #130 (comment)
• start -> movement timestamp
• seconds -> movement amount

[michielbdejong]

We also may want to store the purchase implication rules in the database somehow? There are two parts to that:
we can pretty surely state that some shops delivered something to the payer. But assigning specific budgets is a bit more tricky.

We should import bank statements, expenses, timesheets, and invoices.

It would already be interesting to see all the transactions and their monthly sums, but just show what each entity owes the world, and what their deduplicated transaction log looks like, without separating them out to very specific budgets.

Will try to get that working tomorrow!

[michielbdejong]

I'll refactor the bank statement import so that it gives entries:

• otherComponent (string)
• bankAccountComponent (string)

And keep:

• date -> movement timestamp
• comment -> should go onto the statement somehow?
• amount -> movement amount (positive means bank balance increase / credit, negative means bank balance decrease / debet; after a debet transaction you may go below zero and start paying debet interest)
• balanceAfter -> for sanity checking
• lineNum -> for linking to a line in the doc
• remoteId (ASN only, ING doesn't have this)

That leaves it a bit cleaner how you then use the comment for the statement, the otherComponent for a component, etc

[michielbdejong]

I may still need a better word for statement that makes it clear whether it's a line in a doc, or the underlying belief that may appear in several docs. Maybe statement and object, or statement and notion?
Object is probably better because it refers to an entity (component, movement) that has an identity and not a thesis that can be true or false. It's also more common in programming and although it's usually not meant to mean an object in the memory/db of the current computer, not in the memory/db of another computer, I think it can work.

[michielbdejong]

Ah bummer, feels like this branch has gone stale and I left some work half-done there.
I should try to find time to revisit this, maybe in November or December.