Add option to not append uploaded filename to URL

[kenh]

This adds an option to not automatically append the filename to the supplied URL when uploading a file with PUT.

I ran into this issue when using curl to upload a file to REST API endpoint. I ran into the following confluence of barriers:

• The specific API endpoint ended in a /; you could not omit the final / and when using -T curl would automatically append the filename, the URL ended up being incorrect and would not work.
• The files were very large and you could not use something like --data-binary because that reads the entire file into memory.

I temporarily worked around this by using a rewrite rule on my server, but I don't view that as a long-term solution. I believe I got all of the documentation pieces correct.

[dfandrich]

Can't you use -T - and pass the file on stdin? The api that insists on a slash seems a bit weird.

[kenh]

Can't you use -T - and pass the file on stdin? The api that insists on a slash seems a bit weird.

I am sorry, I forgot to mention that I tried that. That does not work either; because the file size is unknown at that point curl defaults to chunked transfers and those are not supported (the web application is written in Python and it specifically threw an error on the server saying chunked transfers were not supported). I THEN tried adding a Content-Length header with the correct value and adding the header Transfer-Encoding: identity and that also threw an error on the server. I suppose this could also be addressed by doing fstat() on stdin and if that succeeds then filling in the file size.

[emanuele6]

I think that's a bug.

curl does not to try to call fstat() to learn the size of stdin even though stdin is a regular file:

$ curl -vsSo /dev/null --url example.org -T ~/.bashrc
*   Trying 93.184.216.34:80...
* Connected to example.org (93.184.216.34) port 80 (#0)
> PUT /.bashrc HTTP/1.1
> Host: example.org
> User-Agent: curl/8.1.2
> Accept: */*
> Content-Length: 776
> 
} [776 bytes data]
* We are completely uploaded and fine
< HTTP/1.1 404 Not Found
< Cache-Control: max-age=604800
< Content-Type: text/html; charset=UTF-8
< Date: Thu, 29 Jun 2023 19:26:02 GMT
< Expires: Thu, 06 Jul 2023 19:26:02 GMT
< Server: EOS (vny/0451)
< Vary: Accept-Encoding
< Content-Length: 1256
< 
{ [1256 bytes data]
* Connection #0 to host example.org left intact

$ curl -vsSo /dev/null --url example.org -T - < ~/.bashrc             
*   Trying 93.184.216.34:80...
* Connected to example.org (93.184.216.34) port 80 (#0)
> PUT / HTTP/1.1
> Host: example.org
> User-Agent: curl/8.1.2
> Accept: */*
> Transfer-Encoding: chunked
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
} [783 bytes data]
* Signaling end of chunked upload via terminating chunk.
} [5 bytes data]
**hangs forever**

If that is fixed, you should be able to use -T - < "$file" (n.b. not cat -- "$file" | curl -T - ...) to upload the file with content-length.

[kenh]

If that is fixed, you should be able to use -T - < "$file" (n.b. not cat -- "$file" | curl -T - ...) to upload the file with content-length.

I agree! But it also seems kind of non-optimal that curl adds stuff to the specified URL when I do not want it to. I understand why it does that, and I agree that should remain the default AND that the API I am using is kind of weird. But as far as I can tell this is a legitimate case where curl should not do that.

[bagder]

the API I am using is kind of weird. But as far as I can tell this is a legitimate case where curl should not do that.

Maybe --request-target can be used and be enough for this edge case?

[kenh]

Maybe --request-target can be used and be enough for this edge case?

Ah, I did not know about that option! I just tried it and that works. So I guess that's an acceptable alternative to this patch.

[bagder]

I guess that's an acceptable alternative to this patch.

I think using what is already present and working is quite a lot better than adding new functionality. Let's stick with that.