Detect permissible passive endpoints via authentication info
Manually maintaining a list of allowed passive endpoints as suggested in #2 (closed) is difficult in federated infrastructures. One idea for automatic detection of permissible endpoints would be to check that the user-supplied authentication is both required and accepted at the endpoint.
A pre-check before any transfer activities would verify these properties.
- A harmless request to the endpoint without authentication credentials is performed. This request should fail with a 401 response, indicating that authentication is required.
- The same request is performed again with authentication credentials. This should either succeed or at least indicate that the credentials are accepted. Instead of this request, the transfer could also be started directly, expecting that either the credentials are correct or that the endpoints returns an authentication-related error (401 or 403 response).
Challenges:
- What is a harmless request?
OPTIONS
would be the best, but may not require authentication and may also not be fully implemented for storage services.HEAD
is another option, which is widely supported conventionally should not trigger significant work on the endpoint. - How to ensure that the pre-check requests trigger the same logic as the actual data transfer requests, i.e. are representative.
A
HEAD
request to the specific source/destination may be the best general solution here. Still implementation details vary between endpoint software. There is also the challenge of distinguishing the bad-authentication response from a good-authentication-but-another-problem response, e.g. a 404. - Some endpoints may allow both authenticated and unauthenticated requests to some paths. The pre-check would not work here.
/cc @mozhdeh.farhadi