Credentials and connection pools

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Anne van Kesteren
On Mon, May 8, 2017 at 2:28 PM, Patrick McManus <[hidden email]> wrote:
> On Mon, May 8, 2017 at 3:36 AM, Anne van Kesteren <[hidden email]> wrote:
>> A connection can become anonymous-request-tainted when you send an
>> anonymous request over it. If a connection is authenticated the client won't
>> send an anonymous request over it and instead open a new connection.
>
> In at least the common case, cookies, there is just no need for this rule
> and it leads to most of the confusion we have. The connection can easily mix
> cookies and no-cookies.

Agreed. I don't know why you think what I wrote touches that? I'm only
talking about authenticated connections (not requests with
credentials) and requests without credentials (anonymous requests) as
it was my understanding that we did not want to mix those.


--
https://annevankesteren.nl/
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Eric Rescorla
In reply to this post by Anne van Kesteren
On Mon, May 8, 2017 at 5:17 AM, Anne van Kesteren <[hidden email]> wrote:

> On Mon, May 8, 2017 at 1:52 PM, Eric Rescorla <[hidden email]> wrote:
> > On Mon, May 8, 2017 at 12:36 AM, Anne van Kesteren <[hidden email]>
> wrote:
> >> How is that sufficient? How does Servo derive an implementation strategy
> >> from that that's not different from that of other browsers in ways that
> can
> >> be observed by web sites?
> >
> > Way is this an imperative? I should think the requirement was merely
> that it
> > not break Web sites.
>
> And if that requirement is to remain true, we should document what
> sites can come to depend upon.


Sure, but that's a much weaker invariant than cannot be observed.



> And likely will depend on given how
> HTTP/2 push works. Part of the reason I'm raising this is because
> folks are running into issues with the current model. If we change the
> model but don't make sure that all browsers adopt it, developers will
> still end up running into issues.
>

Hmm... Can you describe more about what issues developers of Web sites
are running into?



> >> When a connection is anonymous-request-tainted and the server tries to
> >> authenticate the connection, the client fails that process. This would
> be a
> >> potentially breaking change.
> >
> > I know that this is what McManus was suggesting, but if you mean that
> fail
> > causes the fetch to fail, that seems unnecessary. We could just simulate
> > network error and retry on a different, non-tainted, connection I should
> > think.
>
> We cannot always retry. E.g., with fetch() using streams the user
> agent cannot reproduce the body (by design, to use less memory;
> therefore also fails on most redirects and HTTP authentication).
>

Hmm.... This seems like a separate topic, but as you describe it, this has
the
potential to interact very badly with TLS 1.3 0-RTT (and QUIC too) which
both
depend on the assumption that 0-RTT data can fail and have to be replayed
semantically.

-Ekr
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Patrick McManus
In reply to this post by Anne van Kesteren
On Mon, May 8, 2017 at 8:17 AM, Anne van Kesteren <[hidden email]> wrote:

> >> When a connection is anonymous-request-tainted and the server tries to
> >> authenticate the connection, the client fails that process. This would
> be a
> >> potentially breaking change.
> >
> > I know that this is what McManus was suggesting, but if you mean that
> fail
> > causes the fetch to fail, that seems unnecessary. We could just simulate
> > network error and retry on a different, non-tainted, connection I should
> > think.
>
> We cannot always retry. E.g., with fetch() using streams the user
> agent cannot reproduce the body (by design, to use less memory;
> therefore also fails on most redirects and HTTP authentication).



My mental model is inverted. A connection would be come ineligible for
anonymous requests only when some kind of connection based auth was done to
it (e.g. client certs, or ntlm). The presumption is that this does not
happen often. In its absence there is only one kind of connection and it
can carry anonymous and non anonymous requests.

When some kind of connection based auth is triggered there are 2 classes of
requests that have to deal with this
  a] a single request in flight (this is single because this is an
h1-only-no-pipeline issue)
  b] other requests routed to the same host but not yet written out

I suggest that if [a] is an anonymous request it needs to fail because [a]
triggered the authentication. We can retry it on a clean connection, but
because it itself is the trigger I don't see why that would make any
progress. Maybe I'm misundertanding ekr here.

members of [b] that are anonymous simply need to be late-bound to a
different connection. They haven't been tried yet and aren't failed
(perhaps that was part of the confusion?)
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Anne van Kesteren
In reply to this post by Eric Rescorla
On Mon, May 8, 2017 at 2:33 PM, Eric Rescorla <[hidden email]> wrote:
> On Mon, May 8, 2017 at 5:17 AM, Anne van Kesteren <[hidden email]> wrote:
>> And likely will depend on given how
>> HTTP/2 push works. Part of the reason I'm raising this is because
>> folks are running into issues with the current model. If we change the
>> model but don't make sure that all browsers adopt it, developers will
>> still end up running into issues.
>
> Hmm... Can you describe more about what issues developers of Web sites
> are running into?

One thing that is reported to happen is that the user navigates to /,
then that pushes /x and /y as well. The browser then requests /x
without credentials and doesn't end up using the pushed resource. But
you're probably right that this in essence doesn't depend on the
connection as pushes are request/response bound.

There's also a more general concern that's often aired about using
multiple connections to begin with in an HTTP/2 world, but I'm not
really sure how to qualify that concern. It seems to me that as long
as HTTP/2 goes over TCP, having multiple connections might actually be
beneficial at times.

Nevertheless, connections can be observed and recommending a model for
them doesn't seem like a bad idea to me. Having some kind of contract
as to how they work helps folks who define things like preconnect.
Since if that doesn't know about the full key used for establishing a
connection, you can end up just opening connections that go without
use and neither party really knows why.


>> We cannot always retry. E.g., with fetch() using streams the user
>> agent cannot reproduce the body (by design, to use less memory;
>> therefore also fails on most redirects and HTTP authentication).
>
> Hmm.... This seems like a separate topic, but as you describe it, this has
> the potential to interact very badly with TLS 1.3 0-RTT (and QUIC too) which
> both depend on the assumption that 0-RTT data can fail and have to be replayed
> semantically.

Ouch. Filed https://github.com/whatwg/fetch/issues/538.


--
https://annevankesteren.nl/
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Patrick McManus
In reply to this post by Eric Rescorla
On Mon, May 8, 2017 at 8:33 AM, Eric Rescorla <[hidden email]> wrote:

>
>
> > And likely will depend on given how
> > HTTP/2 push works. Part of the reason I'm raising this is because
> > folks are running into issues with the current model. If we change the
> > model but don't make sure that all browsers adopt it, developers will
> > still end up running into issues.
> >
>
> Hmm... Can you describe more about what issues developers of Web sites
> are running into?



I would encourage us not to take this thread in that direction. h2 push
impls (ours and chrome's) currently scope push to be connection based
rather than writing directly into the cache - and you can see why
connection management would play a role with that little fact of life. But
that algorithm is really just a conservative holding pattern to figure out
the right scope of pushed info. And frankly push is huge amounts of
machinery and corner cases for rather marginal benefits, so I would much
rather fit it into whatever anonymous model we want, rather than working
the other way around.

sleevi has a thread open somewhere about his concerns about not just
sticking pushed data in the cache.. and I have some concerns about what to
do with dynamic data (all pushed data can be consumed once, even if
uncachable.. which makes a lot of sense but then if the http cache is your
backing store how to you make sure more than one copy ends up in the right
place, etc..)

anyhow, that's probably not the right direction for this thread..
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Anne van Kesteren
In reply to this post by Patrick McManus
On Mon, May 8, 2017 at 2:39 PM, Patrick McManus <[hidden email]> wrote:

> On Mon, May 8, 2017 at 8:17 AM, Anne van Kesteren <[hidden email]> wrote:
>> We cannot always retry. E.g., with fetch() using streams the user
>> agent cannot reproduce the body (by design, to use less memory;
>> therefore also fails on most redirects and HTTP authentication).
>
> My mental model is inverted. A connection would be come ineligible for
> anonymous requests only when some kind of connection based auth was done to
> it (e.g. client certs, or ntlm). The presumption is that this does not
> happen often. In its absence there is only one kind of connection and it can
> carry anonymous and non anonymous requests.

Yes, that's exactly what I wrote down. I still don't understand why
you keep saying it's different.


> When some kind of connection based auth is triggered there are 2 classes of
> requests that have to deal with this
>   a] a single request in flight (this is single because this is an
> h1-only-no-pipeline issue)
>   b] other requests routed to the same host but not yet written out
>
> I suggest that if [a] is an anonymous request it needs to fail because [a]
> triggered the authentication. We can retry it on a clean connection, but
> because it itself is the trigger I don't see why that would make any
> progress. Maybe I'm misundertanding ekr here.
>
> members of [b] that are anonymous simply need to be late-bound to a
> different connection. They haven't been tried yet and aren't failed (perhaps
> that was part of the confusion?)

Okay, so instead of failing the connection you fail just the request.
Are you also saying that only HTTP/1 can have authenticated
connections at this point?


--
https://annevankesteren.nl/
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Patrick McManus
On Mon, May 8, 2017 at 8:55 AM, Anne van Kesteren <[hidden email]> wrote:

> Okay, so instead of failing the connection you fail just the request.
> Are you also saying that only HTTP/1 can have authenticated
> connections at this point?
>

I am saying fail the request. The disposition of the connection is a
protocol detail depending on the auth details.. It seems for
TLS-client-auth you would need to fail the connection because the http bits
are stalled mid flight, but for something like NTLM you have a clean
resolution to the auth trigger (it came back with a 401 that we're not
going to act on) and the connection could still be used for other requests.

wrt h1 - yes, I believe right now the only client-authenticated connections
are in h1.
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Patrick McManus
In reply to this post by Eric Rescorla
On Mon, May 8, 2017 at 8:33 AM, Eric Rescorla <[hidden email]> wrote:

> We cannot always retry. E.g., with fetch() using streams the user
> > agent cannot reproduce the body (by design, to use less memory;
> > therefore also fails on most redirects and HTTP authentication).
> >
>
> Hmm.... This seems like a separate topic, but as you describe it, this has
> the
> potential to interact very badly with TLS 1.3 0-RTT (and QUIC too) which
> both
> depend on the assumption that 0-RTT data can fail and have to be replayed
> semantically.



I don't think this is a fetch() level problem - those are both basically
transport issues with unacknowledged data and the transport is always going
to need a copy of that information to accommodate packet loss.. 0-rtt fail
is really just a twist on loss from a replay pov. Our definition of
transport has moved up a bit in the stack from socket buffers, but it
hasn't made it to fetch() yet :)
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Eric Rescorla
On Mon, May 8, 2017 at 6:57 AM, Patrick McManus <[hidden email]>
wrote:

>
> On Mon, May 8, 2017 at 8:33 AM, Eric Rescorla <[hidden email]> wrote:
>
>> We cannot always retry. E.g., with fetch() using streams the user
>> > agent cannot reproduce the body (by design, to use less memory;
>> > therefore also fails on most redirects and HTTP authentication).
>> >
>>
>> Hmm.... This seems like a separate topic, but as you describe it, this has
>> the
>> potential to interact very badly with TLS 1.3 0-RTT (and QUIC too) which
>> both
>> depend on the assumption that 0-RTT data can fail and have to be replayed
>> semantically.
>
>
>
> I don't think this is a fetch() level problem - those are both basically
> transport issues with unacknowledged data and the transport is always going
> to need a copy of that information to accommodate packet loss.. 0-rtt fail
> is really just a twist on loss from a replay pov. Our definition of
> transport has moved up a bit in the stack from socket buffers, but it
> hasn't made it to fetch() yet :)
>

Right. I think in this case it's the semantic layer of HTTP (i.e., the one
that spans h1/h2/h2s/h2q)

-Ekr
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Eric Rescorla
McManus and I talked offline, and I think we agree:

1. It is not currently possible to have two outstanding on the wire
requests with
different auth semantics:

- People don't do H1 pipelining so the outstanding requests can be
late-bound to
the right connection.

- H2 prohibits renegotiation (though we might in future have to worry about
TLS 1.3 post-handshake auth or Nick Sullivan's proposed extensions).


2. With both H1 and H2, you *do* need to store unacknowledged requests so
you can replay them in the face of a variety of transport issues.

-Ekr


On Mon, May 8, 2017 at 7:44 AM, Eric Rescorla <[hidden email]> wrote:

>
>
> On Mon, May 8, 2017 at 6:57 AM, Patrick McManus <[hidden email]>
> wrote:
>
>>
>> On Mon, May 8, 2017 at 8:33 AM, Eric Rescorla <[hidden email]> wrote:
>>
>>> We cannot always retry. E.g., with fetch() using streams the user
>>> > agent cannot reproduce the body (by design, to use less memory;
>>> > therefore also fails on most redirects and HTTP authentication).
>>> >
>>>
>>> Hmm.... This seems like a separate topic, but as you describe it, this
>>> has
>>> the
>>> potential to interact very badly with TLS 1.3 0-RTT (and QUIC too) which
>>> both
>>> depend on the assumption that 0-RTT data can fail and have to be replayed
>>> semantically.
>>
>>
>>
>> I don't think this is a fetch() level problem - those are both basically
>> transport issues with unacknowledged data and the transport is always going
>> to need a copy of that information to accommodate packet loss.. 0-rtt fail
>> is really just a twist on loss from a replay pov. Our definition of
>> transport has moved up a bit in the stack from socket buffers, but it
>> hasn't made it to fetch() yet :)
>>
>
> Right. I think in this case it's the semantic layer of HTTP (i.e., the one
> that spans h1/h2/h2s/h2q)
>
> -Ekr
>
>
>
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Patrick McManus
On Mon, May 8, 2017 at 11:02 AM, Eric Rescorla <[hidden email]> wrote:

> 2. With both H1 and H2, you *do* need to store unacknowledged requests so
> you can replay them in the face of a variety of transport issues.
>


to further clarify - the "you" in that sentence refers to the http
implementation, not the content using xhr/fetch.. and there is some flow
control in fetch streams, so it doesn't have an unbounded commitment.
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Eric Rescorla
Agreed.

-Ekr


On Mon, May 8, 2017 at 8:11 AM, Patrick McManus <[hidden email]>
wrote:

>
> On Mon, May 8, 2017 at 11:02 AM, Eric Rescorla <[hidden email]> wrote:
>
>> 2. With both H1 and H2, you *do* need to store unacknowledged requests so
>> you can replay them in the face of a variety of transport issues.
>>
>
>
> to further clarify - the "you" in that sentence refers to the http
> implementation, not the content using xhr/fetch.. and there is some flow
> control in fetch streams, so it doesn't have an unbounded commitment.
>
>
>
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Anne van Kesteren
In reply to this post by Patrick McManus
On Mon, May 8, 2017 at 3:16 PM, Patrick McManus <[hidden email]> wrote:

> On Mon, May 8, 2017 at 8:55 AM, Anne van Kesteren <[hidden email]> wrote:
>> Okay, so instead of failing the connection you fail just the request.
>> Are you also saying that only HTTP/1 can have authenticated
>> connections at this point?
>
> I am saying fail the request. The disposition of the connection is a
> protocol detail depending on the auth details.. It seems for TLS-client-auth
> you would need to fail the connection because the http bits are stalled mid
> flight, but for something like NTLM you have a clean resolution to the auth
> trigger (it came back with a 401 that we're not going to act on) and the
> connection could still be used for other requests.
>
> wrt h1 - yes, I believe right now the only client-authenticated connections
> are in h1.

So a simple change would be to just start reusing HTTP/2 connections
and leave HTTP/1 alone, but more aggressive seems acceptable too, if
everyone can agree on failing the request for NTLM and failing the
connection for TLS-client-auth. Would the appropriate next step be a
bug against Gecko?


The request retry issue is probably best further discussed here:
https://github.com/whatwg/fetch/issues/538 (thanks Eric!).


As for the HTTP/2 push concerns. There's quite a bit of debate here:
https://github.com/whatwg/fetch/issues/354. Including feedback from
sleevi, mt, and mnot. If you could add your perspective there that
would help I think.


Anything else?


--
https://annevankesteren.nl/
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Patrick McManus
On Mon, May 8, 2017 at 1:30 PM, Anne van Kesteren <[hidden email]> wrote:

> So a simple change would be to just start reusing HTTP/2 connections
> and leave HTTP/1 alone, but more aggressive seems acceptable too, if
> everyone can agree on failing the request for NTLM and failing the
> connection for TLS-client-auth. Would the appropriate next step be a
> bug against Gecko?
>


I think what you describe (as more aggressive) would be a fine gecko
implementation. Hopefully fetch would say something much more generic?
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Anne van Kesteren
On Mon, May 8, 2017 at 7:52 PM, Patrick McManus <[hidden email]> wrote:
> I think what you describe (as more aggressive) would be a fine gecko
> implementation.

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1363284.


> Hopefully fetch would say something much more generic?

Could you explain why it's not desirable to describe this in more
detail? It sounds like you don't agree with my preconnect example and
giving developers a guaranteed contract to write code against, but you
didn't really address it.


--
https://annevankesteren.nl/
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

Patrick McManus
I think saying that anonymous and non-anonymous requests can share a
connection in most cases is fine - but that when connection auth is present
those connections cannot carry anonymous requests. I believe that satisfies
the need without talking about various bits of protocol versioning and
reuse strategies.

In general I think the previous mistake made (in various documents) was
overspecifying things about the transport and this is unwinding it. I
wouldn't unwind it by making it even more specific.

On Tue, May 9, 2017 at 3:18 AM, Anne van Kesteren <[hidden email]> wrote:

> On Mon, May 8, 2017 at 7:52 PM, Patrick McManus <[hidden email]>
> wrote:
> > I think what you describe (as more aggressive) would be a fine gecko
> > implementation.
>
> I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1363284.
>
>
> > Hopefully fetch would say something much more generic?
>
> Could you explain why it's not desirable to describe this in more
> detail? It sounds like you don't agree with my preconnect example and
> giving developers a guaranteed contract to write code against, but you
> didn't really address it.
>
>
> --
> https://annevankesteren.nl/
>
>
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
Reply | Threaded
Open this post in threaded view
|

Re: Credentials and connection pools

blartpaul1337
In reply to this post by Anne van Kesteren
On Sunday, May 7, 2017 at 2:07:39 AM UTC-4, Anne van Kesteren wrote:
> As I understand things we pick connections to reuse based on an origin and a credentials flag (set/unset). This got a little bit more complicated with HTTP/2 as it's not just an origin A, but also any other "origin" entries in A's certificate, but that's not what I'm after.
>
> What I'd like to understand is the history behind using credentials as a key and what we can do to possibly change it. We now have some features that don't send credentials by default (even same-origin), such as <script type=module> and fetch(), and as a result you get two same-origin HTTP/2 connections. This plays poorly with HTTP/2 push.
>
> A standards-related discussion on this is hosted here: https://github.com/whatwg/fetch/issues/341. (Fetch also defines when connections get reused (though it doesn't define the HTTP/2 certificate bits yet).)
>
> (As an aside, having someone from the networking team watch whatwg/fetch and give feedback would really help that work progress faster. Alternatively, if you give me some GitHub IDs to ping when I get stuck that could work too. Much appreciated.)

 ,m
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network
12