threads and server object

I've dug through HttpChannel, etc and I see that I don't need most of
what it is implementing.  And after thinking about it for a while it
doesn't look that complex to build a single user proxy server from new
code. I'll still use HttpChannel to get pages from the remote servers.

So how do I get threading to work? I have three threads active: UI, IO
and timer. I am creating my objects on the UI thread which triggers
the non-thread safe assertion when I use them from the server thread.
How do I create my objects on the server thread? They are completely
asynchronous so blocking is not a problem. I only ever want to use
them from the server thread. Is there an initialization callout from
the server thread that I can use to build my first objects?

--
Jon Smirl
[hidden email]
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

I located my immediate problem, I needed NS_IMPL_THREADSAFE_ISUPPORTS
on one of my objects. That made the assertions go away.

I'm still not 100% clear on what is happening with threading. If I was
building this from scratch I would use a single thread. Everything
would be event driven like lighttpd and polipo. WIth Mozilla/XPCOM it
seems to be a mix of both threads and events.

What is confusing is the nsIServerSocketListener object. It lives on
the UI thread but its events are coming on the socket thread. I
couldn't figure out how to make the nsIServerSocketListener object
live on the socket thread and get rid of the proxy object.

--
Jon Smirl
[hidden email]
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Jon Smirl wrote:
> I'm still not 100% clear on what is happening with threading. If I was
> building this from scratch I would use a single thread. Everything
> would be event driven like lighttpd and polipo. WIth Mozilla/XPCOM it
> seems to be a mix of both threads and events.
>
> What is confusing is the nsIServerSocketListener object. It lives on
> the UI thread but its events are coming on the socket thread. I
> couldn't figure out how to make the nsIServerSocketListener object
> live on the socket thread and get rid of the proxy object.

The design of necko is that threads are an implementation detail. For
channels and server sockets, the idea is that all callbacks happen on
the thread that called asyncOpen/asyncListen. That means that in the
implementation threads and events are used, but these are transparent to
the caller.
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

On 3/4/06, Christian Biesinger <[hidden email]> wrote:
> The design of necko is that threads are an implementation detail. For
> channels and server sockets, the idea is that all callbacks happen on
> the thread that called asyncOpen/asyncListen. That means that in the
> implementation threads and events are used, but these are transparent to
> the caller.

They aren't totally transparent because things like
nsIInputStreamCallback are getting called on the socket thread. I'm
starting to figure this out and get things set up right.

My next problem, when using nsIServerSocketListener how do I tell when
the socket has been closed so I don't loop doing AsyncWait with no
data available.

The header for the available() method says it will return
NS_BASE_STREAM_CLOSED but it doesn't seem to be doing that. I looked
at the source for available() and I can't see where it would return
NS_BASE_STREAM_CLOSED. And in my code it isn't returning
NS_BASE_STREAM_CLOSED, it just returns zero available and I loop.

Is there another mechism for catching a stream close?


--
Jon Smirl
[hidden email]
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Jon Smirl wrote:
> They aren't totally transparent because things like
> nsIInputStreamCallback are getting called on the socket thread. I'm
> starting to figure this out and get things set up right.

Well that depends on the target argument to asyncWait.

> The header for the available() method says it will return
> NS_BASE_STREAM_CLOSED but it doesn't seem to be doing that.

It says it will return that if the stream is closed. It also says that
it returns 0 if the stream if at EOF but not closed. You didn't close it
presumably, so it's not closed.

Hm, the documentation here doesn't seem to be so great...

>  And in my code it isn't returning
> NS_BASE_STREAM_CLOSED, it just returns zero available and I loop.

Sounds like a bug in your code :-) The docs say that 0 means EOF.
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

On 3/4/06, Christian Biesinger <[hidden email]> wrote:

> Jon Smirl wrote:
> > The header for the available() method says it will return
> > NS_BASE_STREAM_CLOSED but it doesn't seem to be doing that.
>
> It says it will return that if the stream is closed. It also says that
> it returns 0 if the stream if at EOF but not closed. You didn't close it
> presumably, so it's not closed.
>
> Hm, the documentation here doesn't seem to be so great...
>
> >  And in my code it isn't returning
> > NS_BASE_STREAM_CLOSED, it just returns zero available and I loop.
>
> Sounds like a bug in your code :-) The docs say that 0 means EOF.

I was basing the code on this. With a zero for EOF you can't tell
whether their isn't data available or if the socket is closed. Zero at
EOF is fine, but it needs to return NS_BASE_STREAM_CLOSED for the
result value.

  /**
     * Determine number of bytes available in the stream.  A non-blocking
     * stream that does not yet have any data to read should return 0 bytes
     * from this method (i.e., it must not throw the NS_BASE_STREAM_WOULD_BLOCK
     * exception).
     *
     * In addition to the number of bytes available in the stream, this method
     * also informs the caller of the current status of the stream.  A stream
     * that is closed will throw an exception when this method is called.  That
     * enables the caller to know the condition of the stream before attempting
     * to read from it.  If a stream is at end-of-file, but not closed, then
     * this method should return 0 bytes available.
     *
     * @return number of bytes currently available in the stream, or
     *   PR_UINT32_MAX if the size of the stream exceeds PR_UINT32_MAX.
     *
     * @throws NS_BASE_STREAM_CLOSED if the stream is closed normally or at
     *   end-of-file
     * @throws <other-error> if the stream is closed due to some error
     *   condition
     */
  /* unsigned long available (); */
  NS_IMETHOD Available(PRUint32 *_retval) = 0;

If I am using a low level input stream from
rv = aTransport->OpenInputStream(0, 0, 0, getter_AddRefs(instream));

Is there any way to peek at the data? If I can't peek at the data I
need to implement my own buffering. With a peek function I can leave
it in the stream buffer. I'm looking for the \n\n\ at the end of the
headers so that I can start header parsing.

--
Jon Smirl
[hidden email]
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Jon Smirl wrote:
> I was basing the code on this. With a zero for EOF you can't tell
> whether their isn't data available or if the socket is closed. Zero at
> EOF is fine, but it needs to return NS_BASE_STREAM_CLOSED for the
> result value.

Hmm. The comments don't match the code.

>      * Determine number of bytes available in the stream.  A non-blocking
>      * stream that does not yet have any data to read should return 0 bytes
>      * from this method (i.e., it must not throw the NS_BASE_STREAM_WOULD_BLOCK
>      * exception).

This disagrees with:
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/netwerk/base/src/nsSocketTransport2.cpp&rev=1.41&mark=285,304#270

:-/

darin, is the code or the documentation wrong here?

> Is there any way to peek at the data? If I can't peek at the data I
> need to implement my own buffering. With a peek function I can leave
> it in the stream buffer. I'm looking for the \n\n\ at the end of the
> headers so that I can start header parsing.

Use ReadSegments, and return a zero read count. That said, that is not
guaranteed to get you all the available data...
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Jon Smirl wrote:
> I was basing the code on this. With a zero for EOF you can't tell
> whether their isn't data available or if the socket is closed. Zero at
> EOF is fine, but it needs to return NS_BASE_STREAM_CLOSED for the
> result value.

Hm, NSPR isn't helping matters:
http://www.mozilla.org/projects/nspr/tech-notes/nonblockingio.html#PR_Available()
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Darin Fisher-2]

On 3/4/06, Christian Biesinger <[hidden email]> wrote:

> Jon Smirl wrote:
> > I was basing the code on this. With a zero for EOF you can't tell
> > whether their isn't data available or if the socket is closed. Zero at
> > EOF is fine, but it needs to return NS_BASE_STREAM_CLOSED for the
> > result value.
>
> Hmm. The comments don't match the code.
>
> >      * Determine number of bytes available in the stream.  A non-blocking
> >      * stream that does not yet have any data to read should return 0 bytes
> >      * from this method (i.e., it must not throw the NS_BASE_STREAM_WOULD_BLOCK
> >      * exception).
>
> This disagrees with:
> http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/netwerk/base/src/nsSocketTransport2.cpp&rev=1.41&mark=285,304#270
>
> :-/
>
> darin, is the code or the documentation wrong here?

The code is wrong IMO.  If you recall, I went over most of the
Available implementations recently and tried to make them match the
documentation.  I guess I missed this one.

-Darin
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Darin Fisher wrote:
> The code is wrong IMO.  If you recall, I went over most of the
> Available implementations recently and tried to make them match the
> documentation.  I guess I missed this one.

Ah, right, I remember now - the way to do this is use readSegments to
determine how many bytes are available.
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

On 3/4/06, Christian Biesinger <[hidden email]> wrote:
> > Is there any way to peek at the data? If I can't peek at the data I
> > need to implement my own buffering. With a peek function I can leave
> > it in the stream buffer. I'm looking for the \n\n\ at the end of the
> > headers so that I can start header parsing.
>
> Use ReadSegments, and return a zero read count. That said, that is not
> guaranteed to get you all the available data...

That goes against this comment in nsIInputStream.h...
 * NOTE: returning NS_OK and (*aWriteCount = 0) has undefined behavior.

/**
 * The signature of the writer function passed to ReadSegments. This
 * is the "consumer" of data that gets read from the stream's buffer.
 *
 * @param aInStream stream being read
 * @param aClosure opaque parameter passed to ReadSegments
 * @param aFromSegment pointer to memory owned by the input stream
 * @param aToOffset amount already read (since ReadSegments was called)
 * @param aCount length of fromSegment
 * @param aWriteCount number of bytes read
 *
 * Implementers should return the following:
 *
 * @return NS_OK and (*aWriteCount > 0) if consumed some data
 * @return <any-error> if not interested in consuming any data
 *
 * Errors are never passed to the caller of ReadSegments.
 *
 * NOTE: returning NS_OK and (*aWriteCount = 0) has undefined behavior.
 *
 * @status FROZEN
 */
typedef NS_CALLBACK(nsWriteSegmentFun)(nsIInputStream *aInStream,
                                       void *aClosure,
                                       const char *aFromSegment,
                                       PRUint32 aToOffset,
                                       PRUint32 aCount,
                                       PRUint32 *aWriteCount);

--
Jon Smirl
[hidden email]
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Jon Smirl wrote:
> That goes against this comment in nsIInputStream.h...
>  * NOTE: returning NS_OK and (*aWriteCount = 0) has undefined behavior.

Well, return an error then. Just make sure not to set *aWriteCount to a
nonzero value.

_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

I'm still trying to figure out how to tell that the socket has been
closed. I was going to use OnTransportEventSink but now I see that it
doesn't have a close event. Seems like it should.

  /**
     * nsITransportEventSink status codes.
     *
     * Although these look like XPCOM error codes and are passed in an nsresult
     * variable, they are *not* error codes.  Note that while they *do* overlap
     * with existing error codes in Necko, these status codes are confined
     * within a very limited context where no error codes may appear, so there
     * is no ambiguity.
     *
     * The values of these status codes must never change.
     *
     * The status codes appear in near-chronological order (not in numeric
     * order).  STATUS_RESOLVING may be skipped if the host does not need to be
     * resolved.  STATUS_WAITING_FOR is an optional status code, which the impl
     * of this interface may choose not to generate.
     */
  enum { STATUS_RESOLVING = 2152398851U };
  enum { STATUS_CONNECTING_TO = 2152398855U };
  enum { STATUS_CONNECTED_TO = 2152398852U };
  enum { STATUS_SENDING_TO = 2152398853U };
  enum { STATUS_WAITING_FOR = 2152398858U };
  enum { STATUS_RECEIVING_FROM = 2152398854U };

--
Jon Smirl
[hidden email]
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Jon Smirl wrote:
> I'm still trying to figure out how to tell that the socket has been
> closed. I was going to use OnTransportEventSink but now I see that it
> doesn't have a close event. Seems like it should.

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/netwerk/base/src/nsInputStreamPump.cpp&rev=1.21&cvsroot=/cvsroot#523

Use this from OnInputStreamReady.

_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Darin Fisher-2]

You should only treat the status events as UI level notifications.
They aren't meant to be programmatically used to switch state.

You can use nsIInputStream::Available to test if the stream is closed.
 (If it returns NS_BASE_STREAM_WOULD_BLOCK, then just treat that as
meaning that the stream is not closed yet.)

By the way, when reading from a socket input stream, you are most
likely reading from a pipe input stream instead.

-Darin



On 3/4/06, Jon Smirl <[hidden email]> wrote:

> I'm still trying to figure out how to tell that the socket has been
> closed. I was going to use OnTransportEventSink but now I see that it
> doesn't have a close event. Seems like it should.
>
>   /**
>      * nsITransportEventSink status codes.
>      *
>      * Although these look like XPCOM error codes and are passed in an nsresult
>      * variable, they are *not* error codes.  Note that while they *do* overlap
>      * with existing error codes in Necko, these status codes are confined
>      * within a very limited context where no error codes may appear, so there
>      * is no ambiguity.
>      *
>      * The values of these status codes must never change.
>      *
>      * The status codes appear in near-chronological order (not in numeric
>      * order).  STATUS_RESOLVING may be skipped if the host does not need to be
>      * resolved.  STATUS_WAITING_FOR is an optional status code, which the impl
>      * of this interface may choose not to generate.
>      */
>   enum { STATUS_RESOLVING = 2152398851U };
>   enum { STATUS_CONNECTING_TO = 2152398855U };
>   enum { STATUS_CONNECTED_TO = 2152398852U };
>   enum { STATUS_SENDING_TO = 2152398853U };
>   enum { STATUS_WAITING_FOR = 2152398858U };
>   enum { STATUS_RECEIVING_FROM = 2152398854U };
>
> --
> Jon Smirl
> [hidden email]
> _______________________________________________
> dev-tech-network mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-network
>

_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

On 3/6/06, Darin Fisher <[hidden email]> wrote:
> You should only treat the status events as UI level notifications.
> They aren't meant to be programmatically used to switch state.
>
> You can use nsIInputStream::Available to test if the stream is closed.
>  (If it returns NS_BASE_STREAM_WOULD_BLOCK, then just treat that as
> meaning that the stream is not closed yet.)
>
> By the way, when reading from a socket input stream, you are most
> likely reading from a pipe input stream instead.

I am using UNBUFFERED since the buffering didn't seem to help any over
what I had to do in my own code. The header parser in
nsResponseHeaders alters the buffer so I need to make a copy.
rv = aTransport->OpenInputStream(nsITransport::OPEN_UNBUFFERED, 0, 0,
getter_AddRefs(instream));

With UNBUFFERED Available() doesn't return any errors. I have
discovered that if I get into AsyncWait and Available() is zero, then
the socket is closed. I haven't figured out any other way to get into
AsyncWait with zero available.  It would be nice if Available() would
return the socked closed error for this case so that I don't have to
rely on the zero available method.

Another solution would be for me to directly catch the status change
events but I don't think there is an accessible hook for me to do
that.

I'm writing my code mainly by making small modifications a couple
objects in netwerk/protocol/http. I'll post a delta patch when I get
things working. I'm making very few changes to the existing code, most
changes add new methods to the existing objects. If I need to do
anything big I'll build a new external object.

--
Jon Smirl
[hidden email]
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Darin Fisher wrote:
> You can use nsIInputStream::Available to test if the stream is closed.
>  (If it returns NS_BASE_STREAM_WOULD_BLOCK, then just treat that as
> meaning that the stream is not closed yet.)

?
So what about:
http://lxr.mozilla.org/seamonkey/source/xpcom/io/nsIInputStream.idl#106

_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Darin Fisher-2]

As I said above, the implementation is wrong, but it doesn't mean that
you can't code against the implementation ;-)

-Darin


On 3/6/06, Christian Biesinger <[hidden email]> wrote:

> Darin Fisher wrote:
> > You can use nsIInputStream::Available to test if the stream is closed.
> >  (If it returns NS_BASE_STREAM_WOULD_BLOCK, then just treat that as
> > meaning that the stream is not closed yet.)
>
> ?
> So what about:
> http://lxr.mozilla.org/seamonkey/source/xpcom/io/nsIInputStream.idl#106
>
> _______________________________________________
> dev-tech-network mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-network
>

_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Darin Fisher-2]

See https://bugzilla.mozilla.org/show_bug.cgi?id=329559


On 3/6/06, Darin Fisher <[hidden email]> wrote:

> As I said above, the implementation is wrong, but it doesn't mean that
> you can't code against the implementation ;-)
>
> -Darin
>
>
> On 3/6/06, Christian Biesinger <[hidden email]> wrote:
> > Darin Fisher wrote:
> > > You can use nsIInputStream::Available to test if the stream is closed.
> > >  (If it returns NS_BASE_STREAM_WOULD_BLOCK, then just treat that as
> > > meaning that the stream is not closed yet.)
> >
> > ?
> > So what about:
> > http://lxr.mozilla.org/seamonkey/source/xpcom/io/nsIInputStream.idl#106
> >
> > _______________________________________________
> > dev-tech-network mailing list
> > [hidden email]
> > https://lists.mozilla.org/listinfo/dev-tech-network
> >
>

_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network

[Christian Biesinger]

Darin Fisher wrote:
> As I said above, the implementation is wrong, but it doesn't mean that
> you can't code against the implementation ;-)

Well, ok, although it seems likely to me that code relying on it will be
broken in 1.8.1 and 1.9 ;)
_______________________________________________
dev-tech-network mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-network