Landing plan for Webrtc

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Landing plan for Webrtc

Randell Jesup-6
Here's the proposed landing plan for WebRTC.  It will land in stages
over the next month+, as portions as ready and as reviews are completed.

Undoubtedly this will be modified significantly along the way, especially
the guesses as to reviewers.

roc has already seen this; his initial comments are included.

--------------------------------------------------------------
Overview of plan to land webrtc code from Alder to Mozilla-Central:

1st tranche (preffed off in configure.in) - to land ASAP
    Build system changes:
       configure.in
       client.mk
       gyp->makefile converter
    Import script
    media/webrtc/trunk/:
        src
        tools
        build
        test (perhaps minus test/data for now until we enable tests)
        testing
    Total size: ~500K lines of .cc & .h files (plus a few .gyp files and .py files for gyp, etc), plus tests

2nd tranche: (might be swapped with some of 3rd tranche depending on
resources) (June 1-Jun-8)
    getUserMedia with Webrtc backend (for non-mobile)
    (initial) tests for getUserMedia
    flip pref
       Must be green on all devices (even if disabled for some targets
       like Android & B2G)

3rd tranche: (June 29)
    SCTP/DataChannel (netwerk)
    transport service (ICE/TURN and p2p transport from EKR)
    signaling (sipcc)
       This is large, on the order of 200-250K lines.
    libjingle (very partial, minus stuff overlapped by signaling work - mostly talk/phone or base)
        Might not be needed at all, or a very small subset.
    libsrtp (probably as media/libsrtp)
    libyuv (probably as media/libyuv)
    Parts of this will likely land directly.
    more tests

4th tranche (July 13)
    peerconnection DOM work
    more tests


---------------
Reviews needed:

We will NOT be line-by-line reviewing 700+K lines of code.  
This is largely imported code.  We will be importing versions
chosen for stable snapshots by Chrome in order to best leverage
Google's testing and security work.  This also will let us watch
any important bugfixes and security fixes to those 'stable' pulls.
Updates on m-c after landing will be pulled from Chrome stable
revs, but with perhaps a bit more examination of the changes as
the size of the patches goes down.

Updates to third-party code (libyuv, libsrtp) to be handled
in conjunction with webrtc.org updates

We will be doing normal line-by-line reviewing of code we wrote
and modifications to the imported code.



Security reviews: To be negotiated with Security team; done in phases
and leveraging Google's work.
  Most likely soft spots will be DOM and signaling.  Maybe DataChannel.
  We need to tie into Google's security team
  We'll need protocol fuzzing
     Avoid too much overlap with Google's testing
     dveditz indicates they have a contractor in Germany? who has
       experience with fuzzing
     Protocols available for fuzzing:
        RTP/RTCP
        SRTP/SRTCP (libsrtp)
        DTLS
        SCTP/DTLS
        ICE
        STUN
        TURN
        VP8 and OPUS packetization (may not be much there)
        NetEQ/etc (fuzz by modifying jitter and loss)
        JSEP/SDP
  User privacy protection and controlling the attack surface of the
     browser will be important considerations.

Initial landing:
  Make system, internal build tools (gyp->make)
     Reviewers: khuey, glandium?, bsmedberg?, release-eng, others TBD
  Review of the import process (and script)
     Reviewers: ted, myk??, glandium?, others TBD
  gyp file changes to webrtc
     Reviewers: ted, khuey, glandium
  custom Windows capture code
     Reviewer: cpearce (already reviewed except for minor #include issue)
  License review
     Reviewer: gerv
  Overall ok from primary leads of Media, Security, Platform.
     Approvals: roc, dveditz?/akeybl?/TBD, JP?

Flipping pref for getUserMedia():
  Review getUserMedia()
     Reviewers: DOM Team member (khuey?), sicking?
  Review capture-path of media/webrtc/trunk/src
     This is high-level review of the imported code before turning it on
     Reviewers: jesup, derf, ekr, ?
  Review device access code
     audio and video, all main platforms
        This is high-level review of the imported code before turning it on
        Reviewer: TBD
  Review rendering paths for preview
     Reviewers: TBD (fabrice?)

Later:
  Opus support in WebRTC
    Reviewers: jesup, derf, ?

  Review protocol paths at a high level (not sure this makes much sense):
    rtp/rtcp/srtp
        Reviewer: jesup
    congestion
        Reviewer: mcmanus
    JSEP/SDP
        Reviewers: ekr, jesup, ehugg
    jitter buffers/NetEq/etc
    Reviewers: jesup, jean-marc, ?
    AEC (and CNG, etc)
        Reviewers: jean-marc, derf, jesup

  DataChannels
    Reviewers: mcmanus/biesi (netwerk/sctp), jst/? (DOM)

  Review libsrtp and libyuv.
    Open question: do we move them to media/libsrtp & libyuv?  probably
      makes sense
    Note: pure import, no line by line review
    Reviewers: ekr, derf, jesup, graphics team member?

  Signaling:  (JSEP/SDP)
    This is sipcc - ~200K lines, and a fair amount of modifications
    Also, a fair bit of code was added to sipcc after it was
      open-sourced and before it was imported from ikran.  That
      code should get higher scrutiny.
    Reviewers: jesup, ekr, derf, mcmanus(?), ehugg, roc?

  PeerConnection DOM
    Reviewers: khuey, jst?

Updates: We'll take updates from 'stable' branches of Chromium's webrtc
pulls, by pulling the same changesets.  We should watch for changes
applied after they're moved to Chrome, and individually review those.


Land stuff separately (and review) wherever possible, as MediaStreams did.
  DataChannel
  getUserMedia()
  Libraries for transport service

--
Randell Jesup, Mozilla Corp
remove ".news" for personal email
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Landing plan for Webrtc

Axel Hecht
Not really related to the landing plan, but as this is external code:

Does that come with strings that we need to localize, like error
messages or stuff?

Axel
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Landing plan for Webrtc

Wes Garland
In reply to this post by Randell Jesup-6
I'm really looking forward to this, as I already have a "production" WebRTC
application running on Chrome Canary. Would really like a second browser
choice in case Google pushes down a breaking update (which would be totally
reasonable under the Canary release guidelines). We used the getUserMedia()
streaming video stuff to build a "photo booth" application.

I tried figuring out which bugs the WebRTC was implemented in -- it looks
like is being largely developed somewhere else than Bugzilla?

I'm now tracking Bug 749889 - Land necessary portion of media/webrtc for
getUserMedia on m-c.

Wes

On 24 May 2012 05:53, Randell Jesup <[hidden email]> wrote:

> Here's the proposed landing plan for WebRTC.  It will land in stages
> over the next month+, as portions as ready and as reviews are completed.
>
> Undoubtedly this will be modified significantly along the way, especially
> the guesses as to reviewers.
>
> roc has already seen this; his initial comments are included.
>
> --------------------------------------------------------------
> Overview of plan to land webrtc code from Alder to Mozilla-Central:
>
> 1st tranche (preffed off in configure.in) - to land ASAP
>    Build system changes:
>       configure.in
>       client.mk
>       gyp->makefile converter
>    Import script
>    media/webrtc/trunk/:
>        src
>        tools
>        build
>        test (perhaps minus test/data for now until we enable tests)
>        testing
>    Total size: ~500K lines of .cc & .h files (plus a few .gyp files and
> .py files for gyp, etc), plus tests
>
> 2nd tranche: (might be swapped with some of 3rd tranche depending on
> resources) (June 1-Jun-8)
>    getUserMedia with Webrtc backend (for non-mobile)
>    (initial) tests for getUserMedia
>    flip pref
>       Must be green on all devices (even if disabled for some targets
>       like Android & B2G)
>
> 3rd tranche: (June 29)
>    SCTP/DataChannel (netwerk)
>    transport service (ICE/TURN and p2p transport from EKR)
>    signaling (sipcc)
>       This is large, on the order of 200-250K lines.
>    libjingle (very partial, minus stuff overlapped by signaling work -
> mostly talk/phone or base)
>        Might not be needed at all, or a very small subset.
>    libsrtp (probably as media/libsrtp)
>    libyuv (probably as media/libyuv)
>    Parts of this will likely land directly.
>    more tests
>
> 4th tranche (July 13)
>    peerconnection DOM work
>    more tests
>
>
> ---------------
> Reviews needed:
>
> We will NOT be line-by-line reviewing 700+K lines of code.
> This is largely imported code.  We will be importing versions
> chosen for stable snapshots by Chrome in order to best leverage
> Google's testing and security work.  This also will let us watch
> any important bugfixes and security fixes to those 'stable' pulls.
> Updates on m-c after landing will be pulled from Chrome stable
> revs, but with perhaps a bit more examination of the changes as
> the size of the patches goes down.
>
> Updates to third-party code (libyuv, libsrtp) to be handled
> in conjunction with webrtc.org updates
>
> We will be doing normal line-by-line reviewing of code we wrote
> and modifications to the imported code.
>
>
>
> Security reviews: To be negotiated with Security team; done in phases
> and leveraging Google's work.
>  Most likely soft spots will be DOM and signaling.  Maybe DataChannel.
>  We need to tie into Google's security team
>  We'll need protocol fuzzing
>     Avoid too much overlap with Google's testing
>     dveditz indicates they have a contractor in Germany? who has
>       experience with fuzzing
>     Protocols available for fuzzing:
>        RTP/RTCP
>        SRTP/SRTCP (libsrtp)
>        DTLS
>        SCTP/DTLS
>        ICE
>        STUN
>        TURN
>        VP8 and OPUS packetization (may not be much there)
>        NetEQ/etc (fuzz by modifying jitter and loss)
>        JSEP/SDP
>  User privacy protection and controlling the attack surface of the
>     browser will be important considerations.
>
> Initial landing:
>  Make system, internal build tools (gyp->make)
>     Reviewers: khuey, glandium?, bsmedberg?, release-eng, others TBD
>  Review of the import process (and script)
>     Reviewers: ted, myk??, glandium?, others TBD
>  gyp file changes to webrtc
>     Reviewers: ted, khuey, glandium
>  custom Windows capture code
>     Reviewer: cpearce (already reviewed except for minor #include issue)
>  License review
>     Reviewer: gerv
>  Overall ok from primary leads of Media, Security, Platform.
>     Approvals: roc, dveditz?/akeybl?/TBD, JP?
>
> Flipping pref for getUserMedia():
>  Review getUserMedia()
>     Reviewers: DOM Team member (khuey?), sicking?
>  Review capture-path of media/webrtc/trunk/src
>     This is high-level review of the imported code before turning it on
>     Reviewers: jesup, derf, ekr, ?
>  Review device access code
>     audio and video, all main platforms
>        This is high-level review of the imported code before turning it on
>        Reviewer: TBD
>  Review rendering paths for preview
>     Reviewers: TBD (fabrice?)
>
> Later:
>  Opus support in WebRTC
>    Reviewers: jesup, derf, ?
>
>  Review protocol paths at a high level (not sure this makes much sense):
>    rtp/rtcp/srtp
>        Reviewer: jesup
>    congestion
>        Reviewer: mcmanus
>    JSEP/SDP
>        Reviewers: ekr, jesup, ehugg
>    jitter buffers/NetEq/etc
>        Reviewers: jesup, jean-marc, ?
>    AEC (and CNG, etc)
>        Reviewers: jean-marc, derf, jesup
>
>  DataChannels
>    Reviewers: mcmanus/biesi (netwerk/sctp), jst/? (DOM)
>
>  Review libsrtp and libyuv.
>    Open question: do we move them to media/libsrtp & libyuv?  probably
>      makes sense
>    Note: pure import, no line by line review
>    Reviewers: ekr, derf, jesup, graphics team member?
>
>  Signaling:  (JSEP/SDP)
>    This is sipcc - ~200K lines, and a fair amount of modifications
>    Also, a fair bit of code was added to sipcc after it was
>      open-sourced and before it was imported from ikran.  That
>      code should get higher scrutiny.
>    Reviewers: jesup, ekr, derf, mcmanus(?), ehugg, roc?
>
>  PeerConnection DOM
>    Reviewers: khuey, jst?
>
> Updates: We'll take updates from 'stable' branches of Chromium's webrtc
> pulls, by pulling the same changesets.  We should watch for changes
> applied after they're moved to Chrome, and individually review those.
>
>
> Land stuff separately (and review) wherever possible, as MediaStreams did.
>  DataChannel
>  getUserMedia()
>  Libraries for transport service
>
> --
> Randell Jesup, Mozilla Corp
> remove ".news" for personal email
> _______________________________________________
> dev-planning mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-planning
>



--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Landing plan for Webrtc

Randell Jesup-6
In reply to this post by Randell Jesup-6
>I'm really looking forward to this, as I already have a "production" WebRTC
>application running on Chrome Canary. Would really like a second browser
>choice in case Google pushes down a breaking update (which would be totally
>reasonable under the Canary release guidelines). We used the getUserMedia()
>streaming video stuff to build a "photo booth" application.
>
>I tried figuring out which bugs the WebRTC was implemented in -- it looks
>like is being largely developed somewhere else than Bugzilla?
>
>I'm now tracking Bug 749889 - Land necessary portion of media/webrtc for
>getUserMedia on m-c.

See bug 'webrtc'; master tracking bug, and view the dependency tree.
(bug 665909)

--
Randell Jesup, Mozilla Corp
remove ".news" for personal email
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Landing plan for Webrtc

Randell Jesup-6
In reply to this post by Axel Hecht
>Not really related to the landing plan, but as this is external code:
>
>Does that come with strings that we need to localize, like error messages
>or stuff?

No, since that's all in the chrome code that will go on top of this.
(I'll verify, but there shouldn't be any I can think of.)

--
Randell Jesup, Mozilla Corp
remove ".news" for personal email
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Landing plan for Webrtc

Randell Jesup-6
In reply to this post by Randell Jesup-6
>Here's the proposed landing plan for WebRTC.  It will land in stages
>over the next month+, as portions as ready and as reviews are completed.
>
>Undoubtedly this will be modified significantly along the way, especially
>the guesses as to reviewers.

So far this plan is largely intact except for dates.  As mentioned in
the Platform Weekly Tuesday, landing is schedules for early morning EDT
today (Wednesday) on inbound to minimize disruptions and easy
identification of any problems.  Note this will likely bloat/slow your
next m-i/m-c pulls.

Patches are all reviewed and staged; try builds are green both with
webrtc disabled (the default) and enabled.  Roc and dveditz both are ok
with landing; I assume JPR is per the Platform Weekly meeting.

disable-webrtc (default):
https://tbpl.mozilla.org/?tree=Try&rev=bd60359fa967

enable-webrtc:
all platforms except linux (one mod agreed to was missing on linux):
https://tbpl.mozilla.org/?tree=Try&rev=70a9cdddf4d1
linux (with the mod):
https://tbpl.mozilla.org/?tree=Try&rev=ed56ccd4d858 and
https://tbpl.mozilla.org/?tree=Try&rev=8d912df99dbc

>roc has already seen this; his initial comments are included.
>
>--------------------------------------------------------------
>Overview of plan to land webrtc code from Alder to Mozilla-Central:
>
>1st tranche (preffed off in configure.in) - to land ASAP
>    Build system changes:
>       configure.in
>       client.mk
>       gyp->makefile converter
>    Import script
>    media/webrtc/trunk/:
>        src
> tools
> build
> test (perhaps minus test/data for now until we enable tests)
> testing
>    Total size: ~500K lines of .cc & .h files (plus a few .gyp files
>    and .py files for gyp, etc), plus tests

One change is that media/webrtc/trunk/third_party/libyuv is landing with
tranche 1 because a bunch of the core video processing code in trunk/src
depends on it.

>
>2nd tranche: (might be swapped with some of 3rd tranche depending on
>resources) (June 1-Jun-8)
>    getUserMedia with Webrtc backend (for non-mobile)
>    (initial) tests for getUserMedia
>    flip pref
>       Must be green on all devices (even if disabled for some targets
>       like Android & B2G)

An initial getUserMedia() patch is in reviews.  From the above try
builds, I expect to flip the pref (and compile webrtc by default) in the
near future.  That will likely expose some compile/system-dependency
bugs beyond those dealt with for our build/test infrastructure (which is
old enough to be a pretty good test).

This tranche will slip but we hope to minimize the slip as most of this
is ready.

>3rd tranche: (June 29)
>    SCTP/DataChannel (netwerk)
>    transport service (ICE/TURN and p2p transport from EKR)
>    signaling (sipcc)
>       This is large, on the order of 200-250K lines.
>    libjingle (very partial, minus stuff overlapped by signaling work - mostly talk/phone or base)
>        Might not be needed at all, or a very small subset.
>    libsrtp (probably as media/libsrtp)
>    libyuv (probably as media/libyuv)
>    Parts of this will likely land directly.
>    more tests

The 3rd tranche (signaling in particular) will need *some* solution to
PGO and gkmedia issues - it has to interact deeply with libxul
(PeerConnections and MediaStreams/Tracks in content/media), and with the
webrtc media code on the gkmedia side.  We're working with
glandium/ted/ehsan (and anyone else interested - khuey I'd assume) on
some options, starting with a static library for gkmedia built with PGO
and without PGO (on Windows).  A backup solution would be to move
signaling to libxul and deal with the pain of exporting a slew of C++
interfaces from gkmedia.

We found the current gkmedia limitations causing huge problems for
the signaling work, and if not solved in some way will delay completion
and add serious amounts of effort, as well as make the resultant code
fragile and hard to maintain.

This tranche will definitely slip; amount TBD, partly dependent on the
issue discussed above.

>4th tranche (July 13)
>    peerconnection DOM work
>    more tests

And this will likely slip; no new date set yet.

>
>
>---------------
>Reviews needed:
>
>We will NOT be line-by-line reviewing 700+K lines of code.  
>This is largely imported code.  We will be importing versions
>chosen for stable snapshots by Chrome in order to best leverage
>Google's testing and security work.  This also will let us watch
>any important bugfixes and security fixes to those 'stable' pulls.
>Updates on m-c after landing will be pulled from Chrome stable
>revs, but with perhaps a bit more examination of the changes as
>the size of the patches goes down.
>
>Updates to third-party code (libyuv, libsrtp) to be handled
>in conjunction with webrtc.org updates
>
>We will be doing normal line-by-line reviewing of code we wrote
>and modifications to the imported code.
>
>
>
>Security reviews: To be negotiated with Security team; done in phases
>and leveraging Google's work.
>  Most likely soft spots will be DOM and signaling.  Maybe DataChannel.
>  We need to tie into Google's security team
>  We'll need protocol fuzzing
>     Avoid too much overlap with Google's testing
>     dveditz indicates they have a contractor in Germany? who has
>       experience with fuzzing
>     Protocols available for fuzzing:
>        RTP/RTCP
> SRTP/SRTCP (libsrtp)
> DTLS
> SCTP/DTLS
> ICE
> STUN
> TURN
> VP8 and OPUS packetization (may not be much there)
> NetEQ/etc (fuzz by modifying jitter and loss)
> JSEP/SDP
>  User privacy protection and controlling the attack surface of the
>     browser will be important considerations.
>
>Initial landing:
>  Make system, internal build tools (gyp->make)
>     Reviewers: khuey, glandium?, bsmedberg?, release-eng, others TBD
>  Review of the import process (and script)
>     Reviewers: ted, myk??, glandium?, others TBD
>  gyp file changes to webrtc
>     Reviewers: ted, khuey, glandium
>  custom Windows capture code
>     Reviewer: cpearce (already reviewed except for minor #include issue)
>  License review
>     Reviewer: gerv
>  Overall ok from primary leads of Media, Security, Platform.
>     Approvals: roc, dveditz?/akeybl?/TBD, JP?
>
>Flipping pref for getUserMedia():
>  Review getUserMedia()
>     Reviewers: DOM Team member (khuey?), sicking?
>  Review capture-path of media/webrtc/trunk/src
>     This is high-level review of the imported code before turning it on
>     Reviewers: jesup, derf, ekr, ?
>  Review device access code
>     audio and video, all main platforms
>        This is high-level review of the imported code before turning it on
>        Reviewer: TBD
>  Review rendering paths for preview
>     Reviewers: TBD (fabrice?)
>
>Later:
>  Opus support in WebRTC
>    Reviewers: jesup, derf, ?
>
>  Review protocol paths at a high level (not sure this makes much sense):
>    rtp/rtcp/srtp
> Reviewer: jesup
>    congestion
> Reviewer: mcmanus
>    JSEP/SDP
> Reviewers: ekr, jesup, ehugg
>    jitter buffers/NetEq/etc
>     Reviewers: jesup, jean-marc, ?
>    AEC (and CNG, etc)
>        Reviewers: jean-marc, derf, jesup
>
>  DataChannels
>    Reviewers: mcmanus/biesi (netwerk/sctp), jst/? (DOM)
>
>  Review libsrtp and libyuv.
>    Open question: do we move them to media/libsrtp & libyuv?  probably
>      makes sense
>    Note: pure import, no line by line review
>    Reviewers: ekr, derf, jesup, graphics team member?
>
>  Signaling:  (JSEP/SDP)
>    This is sipcc - ~200K lines, and a fair amount of modifications
>    Also, a fair bit of code was added to sipcc after it was
>      open-sourced and before it was imported from ikran.  That
>      code should get higher scrutiny.
>    Reviewers: jesup, ekr, derf, mcmanus(?), ehugg, roc?
>
>  PeerConnection DOM
>    Reviewers: khuey, jst?
>
>Updates: We'll take updates from 'stable' branches of Chromium's webrtc
>pulls, by pulling the same changesets.  We should watch for changes
>applied after they're moved to Chrome, and individually review those.
>
>
>Land stuff separately (and review) wherever possible, as MediaStreams did.
>  DataChannel
>  getUserMedia()
>  Libraries for transport service
>
>--
>Randell Jesup, Mozilla Corp
>remove ".news" for personal email


--
Randell Jesup, Mozilla Corp
remove ".news" for personal email
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Landing plan for Webrtc

Wes Garland
On 20 June 2012 02:19, Randell Jesup <[hidden email]> wrote:

> >2nd tranche: (might be swapped with some of 3rd tranche depending on
> >resources) (June 1-Jun-8)
> >    getUserMedia with Webrtc backend (for non-mobile)
> >    (initial) tests for getUserMedia
> >    flip pref
> >       Must be green on all devices (even if disabled for some targets
> >       like Android & B2G)
>
> An initial getUserMedia() patch is in reviews.


I'm guessing you meant July 1-8, here?

Do I understand the bugs (749751 et al) correctly, and the getUserMedia()
support is for still images only, without live preview?

Thanks,
Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning