Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Public Shares CRUD, File Public Shares Manager #681

Merged
merged 50 commits into from
May 28, 2020
Merged

Public Shares CRUD, File Public Shares Manager #681

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
c3e9ef4
set permissions from phoenix
refs Apr 20, 2020
7ac248a
CRUD operations on public shares
refs Apr 23, 2020
41676b3
WIP persistent shares storage
refs Apr 23, 2020
bee35ad
json public share manager
refs Apr 23, 2020
5f8a0df
fix filter, add locally scoped var
refs Apr 23, 2020
a7bb4e6
liberate mutex on defer
refs Apr 24, 2020
9b6cc77
correct time parsing on creation
refs Apr 24, 2020
b7499d5
hide expired public shares
refs Apr 24, 2020
613e636
use correct timestamp on create and update
refs Apr 24, 2020
9cdada6
add comment for future generations
refs Apr 24, 2020
c1cb793
get shares by token working
refs Apr 27, 2020
bb642fa
defer mutex unlock, refactor GetShare
refs Apr 27, 2020
2ad8bf1
use mutex on get public share
refs Apr 27, 2020
8503ee3
code style
refs Apr 27, 2020
6e53187
Merge branch 'master' into public-shares
refs Apr 27, 2020
ac9ab44
address linter
refs Apr 27, 2020
6c84f2a
navigation working
refs Apr 28, 2020
6c118b2
folder navigation on a public share is now supported
refs Apr 28, 2020
a8fe9ee
remove spurious lines
refs Apr 29, 2020
da5a7d2
attempt to create db file if not exists
refs Apr 29, 2020
b160fcd
base64 encode password
refs Apr 29, 2020
52c9d2d
in memory grant manager as a proof of concept
refs May 8, 2020
20dfb8b
remove spurious code
refs May 8, 2020
8bfcc03
make use of err, get rid of ineffectual assignment
refs May 12, 2020
2baa43e
run go mod tidy
refs May 12, 2020
8159d06
Merge branch 'master' into public-shares
refs May 12, 2020
e46fb64
update golangci to 1.26.0
refs May 13, 2020
ef3549a
update golangci-lint counterpart on Makefile
refs May 13, 2020
e071712
force cache invalidation for golangci-lint
refs May 13, 2020
d918fa7
add user to request context, handlePropfind depends on the context user
refs May 14, 2020
9729151
use [public] share manager driver instead of the gateway, as it is an…
refs May 14, 2020
40ab8cb
update golangci-lint on build-onlly and build-and-publish-docker pipe…
refs May 14, 2020
575fee7
ignore jsonpb on golangci.yaml
refs May 14, 2020
4cff37e
get rid of metadata-password-passing hack
refs May 19, 2020
5a0a73a
remove comments
refs May 19, 2020
a7746f0
Merge branch 'master' into public-shares
refs May 19, 2020
9b6adc7
use a db file for link.Grant
refs May 26, 2020
455e6da
fix linter
refs May 26, 2020
58e0b25
Merge branch 'master' into public-shares
refs May 26, 2020
21dbf29
fix propfind stack
refs May 27, 2020
a07b579
defer usage of ref
refs May 27, 2020
39f3fc5
Fix PROPFIND with Depth 1
May 27, 2020
7fd06cf
Merge pull request #1 from PVince81/public-shares
refs May 27, 2020
72fba4b
Merge branch 'master' into public-shares
refs May 28, 2020
68c1932
publicshare: commit grant
labkode May 28, 2020
f395902
Merge pull request #2 from labkode/refs-hugo
refs May 28, 2020
df39963
adapt db json file marshal / unmarshal
refs May 28, 2020
d6eaa1c
remove deadcode
refs May 28, 2020
91bf637
remove yet more dead code
refs May 28, 2020
e42564b
remove unused config parameter
refs May 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 71 additions & 4 deletions internal/grpc/services/gateway/publicshareprovider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,61 @@ func (s *svc) CreatePublicShare(ctx context.Context, req *link.CreatePublicShare
return nil, err
}

// Grants are not accessible through a link.PublicShare, in order for a service to access a grant (link.Grant) this needs to be
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIP. We need to persist the grant for basic authentication password checking. Still need to figure out whether to commit it to the storage or find another way

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @refs I don't think we need to commit the grant to storage for a public share, keeping it outside the storage is good enough. User accessing a public link they cannot access the storage directly anyway. What can be stored in the storage is a reference to the public link ID as metadata.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes a clean separation of concerns. Currently the password is stored on a link.Grant, that is required at the moment of a PublicShare creation, but the PublicShare itself does not include a Grant, by design. So in short, where would you think it would be a good way to store this? We were thinking on the public shares auth provider.

/cc @butonic

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bump. This is the only opinionated bit that needs to be resolved before having a functional persistent public shares storage

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the time being I'm running with an in-memory map for the lack of decision here. This should NOT be adopted in production, but as a proof of concept does the job on checking Basic credentials with the stored public share grants.

// commited to the Storage, for later retrievals. The following intends to do just that.

// TODO(labkode): if both commits are enabled they could be done concurrently.
// if s.c.CommitShareToStorageGrant {
// raw, err := json.Marshal(req.Grant)
// if err != nil {
// return nil, err
// }

// grantReq := &provider.AddGrantRequest{
// Ref: &provider.Reference{
// Spec: &provider.Reference_Id{
// Id: req.ResourceInfo.Id,
// },
// },
// Grant: &provider.Grant{
// Permissions: req.Grant.Permissions.Permissions,
// },
// Opaque: &typesv1beta1.Opaque{
// Map: map[string]*typesv1beta1.OpaqueEntry{
// "grant": &typesv1beta1.OpaqueEntry{
// Decoder: "json",
// Value: raw,
// },
// },
// },
// }

// log.Info().Interface("grantReq", grantReq).Msg("commiting share to storage grant")

// c, err := s.findByID(ctx, req.ResourceInfo.Id)
// if err != nil {
// if _, ok := err.(errtypes.IsNotFound); ok {
// return &link.CreatePublicShareResponse{
// Status: status.NewNotFound(ctx, "storage provider not found"),
// }, nil
// }
// return &link.CreatePublicShareResponse{
// Status: status.NewInternal(ctx, err, "error finding storage provider"),
// }, nil
// }

// grantRes, err := c.AddGrant(ctx, grantReq)
// if err != nil {
// return nil, errors.Wrap(err, "gateway: error calling AddGrant")
// }
// if grantRes.Status.Code != rpc.Code_CODE_OK {
// return &link.CreatePublicShareResponse{
// Status: status.NewInternal(ctx, status.NewErrorFromCode(grantRes.Status.Code, "gateway"),
// "error committing share to storage grant"),
// }, nil
// }
// }

// TODO(refs) commit to storage if configured
return res, nil
}
Expand Down Expand Up @@ -74,6 +129,7 @@ func (s *svc) GetPublicShareByToken(ctx context.Context, req *link.GetPublicShar
}

func (s *svc) GetPublicShare(ctx context.Context, req *link.GetPublicShareRequest) (*link.GetPublicShareResponse, error) {
// TODO if the date is expired DO NOT return
log := appctx.GetLogger(ctx)
log.Info().Msg("get public share")

Expand Down Expand Up @@ -106,18 +162,29 @@ func (s *svc) ListPublicShares(ctx context.Context, req *link.ListPublicSharesRe

res, err := pClient.ListPublicShares(ctx, req)
if err != nil {
return nil, errors.Wrap(err, "error calling ListShares")
return nil, errors.Wrap(err, "error listing shares")
}

return res, nil
}

func (s *svc) UpdatePublicShare(ctx context.Context, req *link.UpdatePublicShareRequest) (*link.UpdatePublicShareResponse, error) {
log := appctx.GetLogger(ctx)
log.Info().Msg("list public share")
log.Info().Msg("update public share")

res := &link.UpdatePublicShareResponse{
Status: status.NewOK(ctx),
pClient, err := pool.GetPublicShareProviderClient(s.c.PublicShareProviderEndpoint)
if err != nil {
log.Err(err).Msg("error connecting to a public share provider")
return &link.UpdatePublicShareResponse{
Status: &rpc.Status{
Code: rpc.Code_CODE_INTERNAL,
},
}, nil
}

res, err := pClient.UpdatePublicShare(ctx, req)
if err != nil {
return nil, errors.Wrap(err, "error updating share")
}
return res, nil
}
26 changes: 23 additions & 3 deletions internal/grpc/services/gateway/storageprovider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -376,10 +376,30 @@ func (s *svc) initiateFileUpload(ctx context.Context, req *provider.InitiateFile
}

func (s *svc) GetPath(ctx context.Context, req *provider.GetPathRequest) (*provider.GetPathResponse, error) {
res := &provider.GetPathResponse{
Status: status.NewUnimplemented(ctx, nil, "GetPath not yet implemented"),
ref := &provider.Reference{
Spec: &provider.Reference_Id{
Id: req.ResourceId,
},
}
return res, nil

statReq := &provider.StatRequest{
Ref: ref,
}
res, err := s.stat(ctx, statReq)
if err != nil {
err = errors.Wrap(err, "gateway: error stating ref:"+ref.String())
return nil, err
}

if res.Status.Code != rpc.Code_CODE_OK {
err := status.NewErrorFromCode(res.Status.Code, "gateway")
return nil, err
}

return &provider.GetPathResponse{
Status: res.Status,
Path: res.GetInfo().GetPath(),
}, nil
}

func (s *svc) CreateContainer(ctx context.Context, req *provider.CreateContainerRequest) (*provider.CreateContainerResponse, error) {
Expand Down
33 changes: 27 additions & 6 deletions internal/grpc/services/publicshareprovider/publicshareprovider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ func New(m map[string]interface{}, ss *grpc.Server) (rgrpc.Service, error) {

func (s *service) CreatePublicShare(ctx context.Context, req *link.CreatePublicShareRequest) (*link.CreatePublicShareResponse, error) {
log := appctx.GetLogger(ctx)
log.Info().Msg("create public share")
log.Info().Str("publicshareprovider", "create").Msg("create public share")

u, ok := user.ContextGetUser(ctx)
if !ok {
Expand All @@ -121,7 +121,7 @@ func (s *service) CreatePublicShare(ctx context.Context, req *link.CreatePublicS

func (s *service) RemovePublicShare(ctx context.Context, req *link.RemovePublicShareRequest) (*link.RemovePublicShareResponse, error) {
log := appctx.GetLogger(ctx)
log.Info().Msg("remove public share")
log.Info().Str("publicshareprovider", "remove").Msg("remove public share")

return &link.RemovePublicShareResponse{
Status: status.NewOK(ctx),
Expand All @@ -145,17 +145,27 @@ func (s *service) GetPublicShareByToken(ctx context.Context, req *link.GetPublic

func (s *service) GetPublicShare(ctx context.Context, req *link.GetPublicShareRequest) (*link.GetPublicShareResponse, error) {
log := appctx.GetLogger(ctx)
log.Info().Msg("get public share")
log.Info().Str("publicshareprovider", "get").Msg("get public share")

u, ok := user.ContextGetUser(ctx)
if !ok {
log.Error().Msg("error getting user from context")
}

found, err := s.sm.GetPublicShare(ctx, u, req.Ref)
if err != nil {
return nil, err
}

return &link.GetPublicShareResponse{
Status: status.NewOK(ctx),
// Share: share,
Share: found,
}, nil
}

func (s *service) ListPublicShares(ctx context.Context, req *link.ListPublicSharesRequest) (*link.ListPublicSharesResponse, error) {
log := appctx.GetLogger(ctx)
log.Info().Msg("list public share")
log.Info().Str("publicshareprovider", "list").Msg("list public share")
user, _ := user.ContextGetUser(ctx)

shares, err := s.sm.ListPublicShares(ctx, user, req.Filters, &provider.ResourceInfo{})
Expand All @@ -175,10 +185,21 @@ func (s *service) ListPublicShares(ctx context.Context, req *link.ListPublicShar

func (s *service) UpdatePublicShare(ctx context.Context, req *link.UpdatePublicShareRequest) (*link.UpdatePublicShareResponse, error) {
log := appctx.GetLogger(ctx)
log.Info().Msg("list public share")
log.Info().Str("publicshareprovider", "update").Msg("update public share")

u, ok := user.ContextGetUser(ctx)
if !ok {
log.Error().Msg("error getting user from context")
}

updateR, err := s.sm.UpdatePublicShare(ctx, u, req, nil)
if err != nil {
log.Err(err).Msgf("error updating public shares: %v", err)
}

res := &link.UpdatePublicShareResponse{
Status: status.NewOK(ctx),
Share: updateR,
}
return res, nil
}
69 changes: 44 additions & 25 deletions internal/grpc/services/publicstorageprovider/publicstorageprovider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,21 +146,39 @@ func (s *service) Stat(ctx context.Context, req *provider.StatRequest) (*provide
trace.StringAttribute("ref", req.Ref.String()),
)

refFromReq, err := s.refFromRequest(ctx, req)
tkn, relativePath, err := s.unwrap(ctx, req.Ref)
if err != nil {
return nil, err
}

pathFromToken, err := s.pathFromToken(ctx, tkn)
if err != nil {
return nil, err
}

statResponse, err := s.gateway.Stat(
ctx,
&provider.StatRequest{
Ref: refFromReq,
Ref: &provider.Reference{
Spec: &provider.Reference_Path{
Path: path.Join("/", pathFromToken, relativePath),
},
},
})
if err != nil {
log.Error().Err(err).Msg("error during stat call")
return nil, err
}

// we don't want to leak the path
statResponse.Info.Path = path.Join("/", tkn, relativePath)

// if statResponse.Status.Code != rpc.Code_CODE_OK {
// if statResponse.Status.Code == rpc.Code_CODE_NOT_FOUND {
// // log.Warn().Str("path", refFromToken.GetPath()).Msgf("resource: `%v` not found", refFromToken.GetId())
// }
// }

return statResponse, nil
}

Expand All @@ -169,7 +187,12 @@ func (s *service) ListContainerStream(req *provider.ListContainerStreamRequest,
}

func (s *service) ListContainer(ctx context.Context, req *provider.ListContainerRequest) (*provider.ListContainerResponse, error) {
statResponse, err := s.Stat(ctx, &provider.StatRequest{Ref: req.Ref})
tkn, relativePath, err := s.unwrap(ctx, req.Ref)
if err != nil {
return nil, err
}

pathFromToken, err := s.pathFromToken(ctx, tkn)
if err != nil {
return nil, err
}
Expand All @@ -178,15 +201,20 @@ func (s *service) ListContainer(ctx context.Context, req *provider.ListContainer
ctx,
&provider.ListContainerRequest{
Ref: &provider.Reference{
Spec: &provider.Reference_Id{
Id: statResponse.GetInfo().GetId(),
Spec: &provider.Reference_Path{
Path: path.Join("/", pathFromToken, relativePath),
},
},
},
)
if err != nil {
return nil, err
}

for i := range listContainerR.Infos {
listContainerR.Infos[i].Path = path.Join("/", tkn, relativePath, path.Base(listContainerR.Infos[i].Path))
}

return listContainerR, nil
}

Expand Down Expand Up @@ -272,31 +300,22 @@ func (s *service) trimMountPrefix(fn string) (string, error) {
return "", errors.New(fmt.Sprintf("path=%q does not belong to this storage provider mount path=%q"+fn, s.mountPath))
}

// refFromRequest returns a reference from a public share token.
func (s *service) refFromRequest(ctx context.Context, req *provider.StatRequest) (*provider.Reference, error) {
ctx, span := trace.StartSpan(ctx, "refFromRequest")
defer span.End()

span.AddAttributes(
trace.StringAttribute("ref", req.Ref.String()),
)

token, _, err := s.unwrap(ctx, req.Ref)
if err != nil {
return nil, err
}

// pathFromToken returns a reference from a public share token.
func (s *service) pathFromToken(ctx context.Context, token string) (string, error) {
publicShareResponse, err := s.gateway.GetPublicShareByToken(
ctx,
&link.GetPublicShareByTokenRequest{Token: token},
)
if err != nil {
return nil, err
return "", err
}

return &provider.Reference{
Spec: &provider.Reference_Id{
Id: publicShareResponse.GetShare().ResourceId,
},
}, nil
pathRes, err := s.gateway.GetPath(ctx, &provider.GetPathRequest{
ResourceId: publicShareResponse.GetShare().GetResourceId(),
})
if err != nil {
return "", err
}

return pathRes.Path, nil
}
39 changes: 33 additions & 6 deletions internal/grpc/services/storageprovider/storageprovider.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,15 @@ package storageprovider

import (
"context"
// "encoding/json"
"fmt"
"net/url"
"os"
"path"
"strings"

rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
// link "github.com/cs3org/go-cs3apis/cs3/sharing/link/v1beta1"
provider "github.com/cs3org/go-cs3apis/cs3/storage/provider/v1beta1"
"github.com/cs3org/reva/pkg/appctx"
"github.com/cs3org/reva/pkg/errtypes"
Expand Down Expand Up @@ -657,18 +659,43 @@ func (s *service) ListGrants(ctx context.Context, req *provider.ListGrantsReques
return nil, nil
}

const (
ContextGrant = iota
)

func (s *service) AddGrant(ctx context.Context, req *provider.AddGrantRequest) (*provider.AddGrantResponse, error) {
// check grantee type is valid
if req.Grant.Grantee.Type == provider.GranteeType_GRANTEE_TYPE_INVALID {
newRef, err := s.unwrap(ctx, req.Ref)
if err != nil {
return &provider.AddGrantResponse{
Status: status.NewInvalid(ctx, "grantee type is invalid"),
Status: status.NewInternal(ctx, err, "error unwrapping path"),
}, nil
}
// check if there is an opaque grant
// if req.GetOpaque() != nil {
// linkGrant := link.Grant{}
// val := req.GetOpaque().GetMap()
// if grantFromOpaque, ok := val["grant"]; ok {
// // TODO need to use map decoder.
// json.Unmarshal(grantFromOpaque.Value, &linkGrant)
// }

// err := s.storage.AddGrant(ctx, newRef, nil)
// if err != nil {
// return &provider.AddGrantResponse{
// Status: status.NewInternal(ctx, err, "error setting ACL"),
// }, nil
// }

// res := &provider.AddGrantResponse{
// Status: status.NewOK(ctx),
// }
// return res, nil
// }

newRef, err := s.unwrap(ctx, req.Ref)
if err != nil {
// check grantee type is valid
if req.Grant.Grantee.Type == provider.GranteeType_GRANTEE_TYPE_INVALID {
return &provider.AddGrantResponse{
Status: status.NewInternal(ctx, err, "error unwrapping path"),
Status: status.NewInvalid(ctx, "grantee type is invalid"),
}, nil
}

Expand Down
Loading