An architecture for programmable storage markets #298
Replies: 7 comments 9 replies
-
|
First some smaller comments: type AllocationClaim struct {
Allocation AllocationID
PieceCID cid.Cid
Size abi.PaddedPieceSize
}DataCapAllocation already contains the PieceCID why does it live here as well? type PieceInfo struct {
// The piece data commitment and size, verified by the miner actor.
CID cid.Cid
Size abi.PaddedPieceSize
// Market-specific identifier for this piece, provided by the provider.
// E.g. the deal ID which the provider claims this piece satisfies.
// TODO: can we find a more generic name than "deal"?
// TODO: should we use a more flexibile []byte key?
DealID uint64
}for Market.SectorContentAdded uses a uint64 while for Market.ProveCommitSector uses address.Address (i.e. []byte) I think it should definitely be a []byte, unless uint64 is just a convenient way of restricting the size to 8 bytes.
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the thorough review!
AllocationClaim is a parameter type, rather than state type. The PieceCID in the AllocationClaim is provided by the miner operator when proving a sector. The operator is not trusted. The miner code checks that the piece CID was indeed included in in the sector CommD/R but it can't independently verify that the PieceCID matches the allocation the operator claims it it satisfying. So the PieceCID is passed on to the registry actor to verify. |
Beta Was this translation helpful? Give feedback.
-
Oops, I meant to use uint64 everywhere, thanks. I've left a TODO to consider []byte instead, as you suggest. The flexibility could be good, but a little overhead for the very likely common case of using integers. |
Beta Was this translation helpful? Give feedback.
-
|
Larger comments relating to verified data cap seperation from market actor and the deal flow:
|
Beta Was this translation helpful? Give feedback.
-
There is no revocation mechanism for published but unexpired allocations, by design. If there were, a provider could not trust that the client won't revoke the allocation after doing all the work to fetch and possibly seal the data. This does leave open the possibility of a provider griefing a client a little by leaving the data cap locked until the allocation expires.
One point that may not have been clear: either the client calls AllocateDataCap, or the provider calls PublishAllocations with a certificate provided to it by the client. After either one of those, the provider is ready to claim the allocation by proving the data commitment. In what I expect will be the common case, the provider gets certificates that nominate that provider explicitly and, if acting benevolently, just doesn't publish them if it doesn't intend to claim them. The data cap is never locked, the client is free to re-allocate to another provider. The reason to publish them is to get a timelock on the client's commitment while the provider does the data shuffling. In case the client called AllocateDataCap instead, then yes the allocation would sit around until expiration.
For "no-provider" allocations, I would expect a client to call AllocateDataCap, thus publishing the allocation to the world in one fell swoop. That functions somewhat like a bounty, and at that point it's a race for the first provider to provably commit the data to claim the allocation. I haven't attempted to design the dynamics of how a provider might decide to try to take such a bounty, but I expect there's an equilibrium where they don't try but lose that race very often. A no-provider certificate could be signed and published somewhere else, perhaps to a small group. That would set off the same race. I could be convinced no-provider allocations are a bad idea, but seemed to me to be an expedient way to implement a bounty mechanism.
For all of the deals that are today verified but specify zero storage fee, no market deal is required at all and the provider need never call PublishStorageDeals. The only reason to go to the trouble and cost of publishing a deal is for the client to pay an additional amount, above the quality-adjusted power subsidy. So yes, both your sequencing is accurate (if there is a deal) and also the market interaction sometimes becomes unnecessary. |
Beta Was this translation helpful? Give feedback.
-
|
@hannahhoward both of your concerns ended up solved upstream in the implementation sketch for #313, by the realisation we could model FIL+ data cap as a fungible token and then use the relatively common practise of authorizing delegates to act on behalf of token holders. If the built in market (or other market actor) is authorized as a delegate for FIL+ clients, it can continue to mediate the allocation of data cap so that (1) we don't need explicit pre-allocation and (2) we don't need to change deal negotiation. |
Beta Was this translation helpful? Give feedback.
-
|
I have updated the original post here to reflect a design based on #313 as a prerequisite. The result is much simpler all round. #313 makes all the necessary changes to FIL+. Modelling data cap as a fungible token, with market actors acting as delegates, also removed the need to change off-chain deal negotiation protocol. The result is a relatively small change to the interactions between built-in actors, but the major work ecosystem-wide will be changes to how SPs and clients interact. I want to explore the possibility of making this in a backwards-compatible way, such that we could expose new APIs while leaving the old flows working for a bit. The implementation sketch for #313 does this for data cap, it would be attractive to try to continue it. |
Beta Was this translation helpful? Give feedback.
-
|
FIP-0045 dropped the concept of ranges that can support FIL+ allocations for pieces that are larger than a single sector. One reason for dropping it is that it wouldn't actually be usable until coupled with some API changes like those necessary for this proposal. So when implementing this one, we should consider bringing ranges back into scope here. See #313 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
Another piece initially in FIP-0045 scope but removed was support for re-committing a piece of verified data that was lost, re-claiming the power boost. See #313 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
Updateable sector data, sector expiration changes and movement of data between sectors are all operations that will need to load the sector info. It makes sense to set sector-wide permissions on the sector info based on the piece manifests being committed to. For example a deal that does not want data to stick around past the deal will set We could do this in aggregate at a higher level. For example keep a bunch of permission sets in the miner state. Sectors with immutable data are kept in one bitfield, sectors with data that can move across sectors are kept in another bitfield etc. But since the relevant operations should always load the sector info it's probably better to put permissions there directly. |
Beta Was this translation helpful? Give feedback.
-
|
As long as SectorContentAdded passes over the permissions markets can check these and accept or reject the deal on their side. |
Beta Was this translation helpful? Give feedback.
-
Actually if we go down this route we'll want to revisit this. If we keep it as is then market actors will be unable to gate FIL+ allocations beyond the existing FIL+ checks. It seems like markets should reasonably be able to grant FIL+ to sectors only obeying the correct permissions. |
Beta Was this translation helpful? Give feedback.
-
|
Quick question to help me understand a bit: why is there an execution coupling between the sealing/committing the sector, and the market actors? The pub-sub issue as you mentions is clumsy. Why can't the market come back on behalf of a user, check state, and do something about it via a subsequent on-chain transaction? Having to multi-plex could cause scalability, gas, or execution concerns if somehow the pub/sub on a miner action invoked 100 contracts. Am I understanding this right? |
Beta Was this translation helpful? Give feedback.
-
|
I think you are on the right direction & decoupling miner and market is one of the the main goal of this. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, I think I know where you're getting. Since power management is separate from things like, rewards on deals in user-land, I'm positing that instead of providing an execution-callback to user-land smart contracts, let the smart contracts read the state as part of a separate transaction. Potentially, ensuring more end to end automation might be a task for say, safecron and not having the protocol spend gas to power market contracts when publishing deals. Does that make sense? Maybe its soup. |
Beta Was this translation helpful? Give feedback.
-
|
The reason for the execution coupling is because the commitment to the data in the sector (unsealed CID, aka CommD) is not stored on chain. The only time it is available to actors is in memory during the execution of the ProveCommit or ReplicaUpdate operation. The market can't come back later to check because it's not there. The first answer to why the CommD is not stored on chain is that it's big. Storing CommD for every sector would nearly double the size of the Filecoin state tree; it's dominated already by the similarly sized sector replica commitments (CommR). However, we can do better. @ZenGround0 has some solid ideas in #496, and I expect we'll implement some variant as part of this programmable markets effort. |
Beta Was this translation helpful? Give feedback.
-
July 2022: This proposal has been re-written to take implementation of #313 (with this implementation sketch) as a pre-requisite. This simplifies the remaining changes a good deal. Modelling data cap as a fungible token, with market actors acting as delegates, also removed the need to change off-chain deal negotiation protocol 🎉 . Some of the discussion below thus talk about a more complex change than is now proposed.
Abstract
This proposal is a refactoring of the APIs and interactions between the storage miner actor, built-in storage market actor, and Filecoin Plus verified registry actor to provide a platform for alternative storage markets, incentive schemes, and other sector-content-related applications to be built on the FVM.
The thrust of the architecture is a decoupling and separation of concerns between the three actors. Notable points include:
This proposal requires FIP-0034 simplifying pre-commit deposit as pre-requisite.
This proposal is complemented by miner←→market registrations: if/when we support updatable sector content we would need such a notification mechanism.
Motivation
High level motivation for why to provide a platform for alternative storage markets and other content-related contracts is given in #241.
We need to change the storage market contract architecture because the current structure does not provide the information and hooks that would be needed by new applications to implement storage-deal-like functionality. It privileges the built-in market actor as the one which the storage miner actor consults. In order to enable the development of useful storage-related applications and features, we need to break the tight coupling between these built-in actors, expose composable on-chain primitives, and make the capabilities of the built-in market actor available to user-programmable contracts.
Goals of this architecture proposal include:
It will be more-or-less impossible to remove or change any exposed API after the first user-programmed contracts are deployed to the Filecoin mainnet, without breaking them. Thus, built-in actors should be quite conservative in what they export, as they will be bound to support it for a long time (see #401).
Specification
The following new APIs only include the methods related to storage onboarding and deals. Assume the remainder of these actors’ APIs remain as-is.
This proposal is written assuming that the changes described in FIP-0034 are already in place.
Verified registry actor
See implementation notes in https://www.notion.so/FIL-indefinite-term-limits-08079d7ec25c4cae839a3bf95a82df26
Storage miner actor
State
The miner actor no longer persists deal IDs in state at all.
Methods
Deal IDs are no longer specified at sector pre-commitment. Instead, the pieces of data that comprise a sector’s data are declared as a “manifest” when the replica or update is proven.
The provider must commit to the both the sealed and unsealed CIDs at pre-commit. At PoRep the miner actor then checks that the unsealed CID matches that computed from the piece CIDs (and an optional implicit zero piece for padding), and the PoRep proof verifies it corresponds to the sealed CID also committed earlier. The ReplicaUpdate flow commits and proves at the same time.
Each piece may specify a market contract address and market-specific identifier which it satisfies. The miner will synchronously notify that market address that the piece has been committed. The miner will pay no attention to the success or otherwise of that notification to the market: the sector will be committed regardless. Note that there is no requirement for the actor that is notified to be some kind of marketplace: it is just a contract that is notified when the piece is committed.
Each piece may also declare a FIL+ verified data cap allocation ID that it satisfies. The miner actor will claim that allocation directly from the verified registry actor and, if successful, calculate quality-adjusted power according to the piece’s size.
Storage market actor
The built-in market actor loses its privileged role (except for continued use of the Cron actor), and becomes just one of many possible market mechanisms that could be implemented as user contracts.
The basic
SectorDealsmap in state andSectorContentAddedmethod described below supports deals within the current restrictions of the market actor: deals cannot exceed their sector’s committed lifetime, the data cannot be moved between sectors, deals cannot be extended. Extension methods are outlined to show how this API can be extended to break those limitations in the future.Deal negotiation
The built-in storage market actor is pre-authorized to act as a delegate for verified registry clients, and thus allocates and claims data cap according to the
verifiedfield already present in the deal proposal.The current deal negotiation structures do not have space to communicate an existing data cap allocation ID, but it could be specified out of band.
State
Methods
The market actor becomes an example for other actors. While the deal-negotiation methods need not be standardised, the method/s by which the storage miner actor informs the market of changes to sector content must be standards.
Control & data Flows
The standard end-to-end verified deal onboarding flow currently looks like this:
After this architecture, it looks like this:
A client’s call to
AllocateDataCapis necessary only if they have not delegated a market actor to allocate data cap on their behalf.The storage provider passes the FIL+ allocation, market address, and deal IDs when proving a sector. The miner actor checks that these values match the sector content, and then passes them on as trusted values to the verified registry and market actors.
Frequent flows will be even simpler.
(~nv)(~vo).Migration
A state migration is required to realise this architecture in mainnet. Details to be fleshed out, but in brief:
Discussion
Limitations
Extensions
These APIs are designed with extension to more powerful and flexible programmable storage in mind. In general these will require new methods to be added.
market.SectorExpirationChanged. New methods on miner and/or market actor would be needed to support this.The prior version of this post is available inside this expander.
# AbstractThis proposal is a refactoring of the APIs and interactions between the storage miner actor, built-in storage market actor, and verified registry actor (FIL+) to provide a platform for alternative storage markets, incentive schemes, and other sector-content-related applications to be built on the FVM.
The thrust of the architecture is a decoupling and separation of concerns between the three actors. Notable points include:
This proposal is complementary to miner←→market registrations, and likely requires such a mechanism to safely support multiple markets if/when we support updateable sector content. Changing the pre-commit deposit is also a pre-requisite.
Edits:
Motivation
High level motivation for why to provide a platform for alternative storage markets and other content-related contracts is given in #241.
We need to change the storage market contract architecture because the current structure does not provide the information and hooks that would be needed by new applications to implement storage-deal-like functionality. It privileges the built-in market actor as the one which the storage miner actor consults. In order to enable the development of actually-useful storage-related applications and features, we need to break the tight coupling between these built-in actors, expose composable on-chain primitives, and make the capabilities of the built-in market actor available to user-programmable contracts.
Goals of this architecture proposal include:
It will be more-or-less impossible to remove or change any exposed API after the first user-programmed contracts are deployed to the Filecoin mainnet, without breaking them. Thus, we should be quite conservative in what we expose, as it will bind us for a long time. Please review this proposal with a careful eye for unwanted lock-in that would prohibit reasonably foreseeable use cases.
Specification
I propose first new APIs for the built-in miner, market, and registry actors, followed by a description of the call sequences to achieve the primary use cases. These APIs only include the methods related to storage onboarding and deals. Assume the remainder of these actors’ APIs remain as-is.
This proposal is written assuming no other changes to core mechanisms.
The architecture can be adapted to handle each of these changes. In practise, we should strive to coordinate any change in those policies with implementation of this architecture.
Storage miner actor
State
The miner actor no longer persists deal IDs in state at all.
Methods
Deal IDs are no longer specified at sector pre-commitment. Instead, the pieces of data that comprise a sector’s data are declared as a “manifest” when the replica or update is proven.
The pre-commit deposit must be changed to a value that is independent of sector content. We set it to the 20-day expected reward from a fully-verified sector (i.e. 200-day reward of a CC sector). This is still less that the initial pledge, for all sectors. This is a separate, pre-requisite proposal #290.
The miner must commit to the both the sealed and unsealed CIDs at pre-commit. At PoRep it then checks that the unsealed CID matches that computed from the piece CIDs (and an optional implicit zero piece for padding), and the PoRep/SnapDeals proof verifies it corresponds to the sealed CID also committed earlier.
Each piece may specify a market contract address and market-specific identifier which it satisfies. The miner will synchronously notify that market address that the piece has been committed. The miner will pay no attention to the success or otherwise of that notification to the market: the sector will be committed regardless. Note that there is no requirement for the actor that is so notified to be some kind of marketplace: it is just a contract that is notified when the piece is committed.
Each piece may also declare a FIL+ verified data cap allocation ID that it satisfies. The miner actor will claim that allocation directly from the verified registry actor and, if successful, calculate quality-adjusted power according to the piece’s size. (There is an unresolved question about the term for data cap and associated power boost, omitted here).
Verified registry actor
The verified registry actor now tracks data cap allocations in state explicitly, since it no longer trusts the market actor to do so alongside storage deals. Verified clients allocate data cap to a specific piece of data prior to making any deals for it, or with a signed message attached to a deal. No deal is necessary, though. A verified data allocation may be claimed by a provider that has committed matching data without regard to any storage market arrangements.
The code below does not address the term (time period) for a data cap allocation, which is now independent of any storage market deals. We may add an explicit term or limit, in which case the verified registry actor would likely also maintain state for each active allocation claim.
State
Methods
Verified clients make explicit allocations for data cap with the verified registry actor. A client may make multiple allocations for the same piece (for distinct replicas), but each consume additional data cap.
The miner actor calls to the verified registry actor directly to claim data cap.
Storage market actor
The built-in market actor loses its privileged role (except for continued use of the Cron actor), and becomes just one of many possible market mechanisms that could be implemented as user contracts.
The basic
SectorDealsmap in state andSectorContentAddedmethod described below supports deals within the current restrictions of the market actor: deals cannot exceed their sector’s committed lifetime, the data cannot be moved between sectors, deals cannot be extended. Extension methods are outlined to show how this API can be extended to break those limitations in the future.Deal negotiation
A client must provide information about verified data allocations to the provider, since the (market-specific) deal proposal is no longer reliable. The provider can check with the verified registry actor that an allocation is valid, or publish certificate to the chain either eagerly or lazily.
State
Methods
The market actor becomes an example for other actors. While the deal-negotiation methods need not be standardised, the method/s by which the storage miner actor informs the market of changes to sector content must be standards.
Control & data Flows
The standard end-to-end verified deal onboarding flow currently looks like this:
After this architecture, it looks like this:
The storage provider specifies the FIL+ allocation and deal IDs when proving a sector. The miner actor checks that these values match the sector content, and then passes them on as trusted values to the verified registry and market actors.
Frequent flows will be even simpler.
Migration
A state migration is required to realise this architecture in mainnet. Details to be fleshed out in a FIP, but in brief:
Discussion
Some non-obvious implications of this architecture.
A storage market deal is only necessary when the client intends to pay funds above the block reward boost the provider will earn.
Compatibility
Changes the off-chain deal negotiation protocol to support the client demonstrating a data cap allocation.
Extensions
These APIs are designed with extension to more powerful and flexible programmable storage in mind. In general these will require new methods to be added.
Beta Was this translation helpful? Give feedback.
All reactions