Download module from a specific subdirectory within the repository

[owenhaynes]

Is your feature request related to a problem? Please describe.
Be able to cache a module from a mono-repo which has a vanity URL in front of it.

E.G have a go module in some mixed language environment git repo.
/src/test.cpp
/src/b.cpp
/go/blah.go
/go/go.mod
/go/go.sum

make go.foo.io/foo points to /go/

Describe the solution you'd like
Be able to configure Athens to know how to map a vanity URL to a repo sub folder, avoiding a call to go.foo.io/foo to get the metadata.

Describe alternatives you've considered

1. Creating a custom mod proxy which does this then Athens caches the proxy.
2. Publish to Athens, this means configuring a CI to create a publish job and would require a format to be defined.

Additional context
I had a look to see if I could hack in the solution using the current state but because Athens just calls go mod download we get blocked on the way go mod download works golang/go#33562.

All the other go module proxies I have found all seem to just call go mod download as well.

Go seems to be hiding all the module versioning code in internal packages, unless I have missed to look somewhere. This makes it difficult to reuse there pseudo versioning code and other bits to do with the coderepo, these could just copied but this seems bad. This makes it harder to make Athens not call go mod download and alternative 1.

Alternative 2 seems to much like JAVA and will just pollute the cache

[arschles]

Hi @owenhaynes! discussions on aliasing and vanity URLs have been pretty popular recently, if you're interested. Here are 3 for example: #1315, #1282, and the alias endpoint discussion in #1195.

For your case, just to be clear, does running go get or go mod download on the VCS subdirectory work for you? For example, does GOPROXY=https://athens go get bitbucket.com/foo/foo/src/go work? It looks like that won't solve the problem for you since you need a vanity URL, but I've never tried this myself with non Go code at the top level of the repository, so I want to see if you know before I go try it myself (in theory it should?)

Regarding the vanity URL - I agree with what you said about alternative 2. Allowing folks to publish directly to Athens would make it a point of origin for code and fundamentally push the project beyond being a proxy. I'd like to avoid that. There's room, however, to write code in Athens itself that fetches code and packages it into a valid zip file, without using go mod download.

[owenhaynes]

It was the first thing I tried before seeing if a proxy would solve my problem. The metadata for go get does not support subdirectories golang/go#33562. They don't seem to what to make any changes to the metadata fields. I have managed to hack Athens at the moment to do basic aliases to do the zip business like go get does and support subdirs. Implemented a fetcher which calls git clone and zip routine if its a matches alias and if not calls the standard fetcher. This required a few of the internal go get modules to be copied and modified to support subdirs mostly around walking the git tree with an offset. This seems to be the same code called if you use vanity URLs or not. The most import code from the internal library seems to be around how to generate the versions

[arschles]

The metadata for go get does not support subdirectories

I might be misunderstanding, but what happens if you do a go get directly against a VCS module path, without a vanity domain (and without metadata)?

I have managed to hack Athens at the moment to do basic aliases to do the
zip business like go get does and support subdirs

Do you have this code in a fork somewhere we can take a look? I think there's a lot of interest in seeing how this works and whether we can add it to Athens..

[owenhaynes]

Sorry it taken so long been a bit busy
Git Offset

Like I said this uses a lot of copied code from the go code base, and is no way clean, just a hack.