source history in mozilla2

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

source history in mozilla2

Boris Zbarsky
I'm trying to understand the situation with source history in mozilla2.

As I understand the current setup (which I never saw described anywhere in
public, btw, till it came up on IRC today), we're importing trunk without
history into an hg branch (call it "cvs-trunk").  Then we're merging trunk
changes from CVS to cvs-trunk every 6 hours, with loss of history in the process
(that is, all the changes in those six hours go into hg as a single commit).
The mozilla2 work is happening on some other hg branch (call it "hg-trunk" for
purposes of this discussion), with periodic manual merges from the cvs-trunk
branch to hg-trunk (not sure whether this is a lossy process or not; someone
more familiar with hg would have to comment).

This is fine as a temporary setup, but I'm hoping there is a better plan for
long-term, especially once all work is happening in hg.

To be precise, my use case is needing to find out when a line of code was added
and why.  I need to do this 5-10 times on a typical day.  If we simply take
what's currently set up as hg-trunk and make it the trunk, with no other
changes, this will mean that finding this information would involve both source
repositories, looking at CVS log date ranges to find the possibly right
checkins, etc.  This is a significant burden over the current "cvs annotate and
search" workflow.

I realize that we could possibly hack bonsai to do this behind the scenes, but I
find myself working offline rather often now that I have a laptop.  I'm happy
enough to cart around a local repository to handle history stuff, but carting
around two of them, and manually moving from one to the other all the time, is a
pain.

So I'm hoping there is a plan to deal with this problem.  What is this plan?

-Boris

P.S. I understand there is a feeling in certain quarters that we do not in fact
want "old" source history in the "mozilla2" repository.  I can't describe how
strongly I object to that, for "It'd make working with the thing hellish" reasons.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Jonathan Watt-3
Boris Zbarsky wrote:

> As I understand the current setup (which I never saw described anywhere
> in public, btw, till it came up on IRC today), we're importing trunk
> without history into an hg branch (call it "cvs-trunk").  Then we're
> merging trunk changes from CVS to cvs-trunk every 6 hours, with loss of
> history in the process (that is, all the changes in those six hours go
> into hg as a single commit). The mozilla2 work is happening on some
> other hg branch (call it "hg-trunk" for purposes of this discussion),
> with periodic manual merges from the cvs-trunk branch to hg-trunk (not
> sure whether this is a lossy process or not; someone more familiar with
> hg would have to comment).

That would really suck.

> This is fine as a temporary setup, but I'm hoping there is a better plan
> for long-term, especially once all work is happening in hg.
>
> To be precise, my use case is needing to find out when a line of code
> was added and why.  I need to do this 5-10 times on a typical day.

I often find myself doing that 10 times in as many _minutes_. If the history is
going to be obliterated during the mozilla2 conversion it's going to be a real pain.

> P.S. I understand there is a feeling in certain quarters that we do not
> in fact want "old" source history in the "mozilla2" repository.  I can't
> describe how strongly I object to that, for "It'd make working with the
> thing hellish" reasons.

Seconded! Why would we not want the pre-hg source history? Maybe there are
technical reasons for it not being feasible, but surely we _absolutely_ want it
if its practicable to have it. Having easy access to this info is very valuable.

Jonathan
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Johnny Stenback-3
In reply to this post by Boris Zbarsky
FWIW, I can live w/o bringing in all the history prior to the point
where we decide to start the Mozilla 2 repository, but I agree that
loosing history in the import of changes from CVS from then on would
really suck, a lot.

Because of that, and the fact that there doesn't seem to be a solution
out there that fits our needs, I've been hacking on a script that can
import partial CVS history into a DVCS, producing as sane change sets as
possible etc in the process. It's not beautiful or anything, but it
appears to work pretty good so far. Should be ready for some real
testing in a day or two.

Boris Zbarsky wrote:

> I'm trying to understand the situation with source history in mozilla2.
>
> As I understand the current setup (which I never saw described anywhere
> in public, btw, till it came up on IRC today), we're importing trunk
> without history into an hg branch (call it "cvs-trunk").  Then we're
> merging trunk changes from CVS to cvs-trunk every 6 hours, with loss of
> history in the process (that is, all the changes in those six hours go
> into hg as a single commit). The mozilla2 work is happening on some
> other hg branch (call it "hg-trunk" for purposes of this discussion),
> with periodic manual merges from the cvs-trunk branch to hg-trunk (not
> sure whether this is a lossy process or not; someone more familiar with
> hg would have to comment).
>
> This is fine as a temporary setup, but I'm hoping there is a better plan
> for long-term, especially once all work is happening in hg.
>
> To be precise, my use case is needing to find out when a line of code
> was added and why.  I need to do this 5-10 times on a typical day.  If
> we simply take what's currently set up as hg-trunk and make it the
> trunk, with no other changes, this will mean that finding this
> information would involve both source repositories, looking at CVS log
> date ranges to find the possibly right checkins, etc.  This is a
> significant burden over the current "cvs annotate and search" workflow.
>
> I realize that we could possibly hack bonsai to do this behind the
> scenes, but I find myself working offline rather often now that I have a
> laptop.  I'm happy enough to cart around a local repository to handle
> history stuff, but carting around two of them, and manually moving from
> one to the other all the time, is a pain.
>
> So I'm hoping there is a plan to deal with this problem.  What is this
> plan?
>
> -Boris
>
> P.S. I understand there is a feeling in certain quarters that we do not
> in fact want "old" source history in the "mozilla2" repository.  I can't
> describe how strongly I object to that, for "It'd make working with the
> thing hellish" reasons.


_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Johnny Stenback-3
In reply to this post by Boris Zbarsky
FWIW, I can live w/o bringing in all the history prior to the point
where we decide to start the Mozilla 2 repository, but I agree that
loosing history in the import of changes from CVS from then on would
really suck, a lot.

Because of that, and the fact that there doesn't seem to be a solution
out there that fits our needs, I've been hacking on a script that can
import partial CVS history into a DVCS, producing as sane change sets as
possible etc in the process. It's not beautiful or anything, but it
appears to work pretty good so far. Should be ready for some real
testing in a day or two.

Boris Zbarsky wrote:

> I'm trying to understand the situation with source history in mozilla2.
>
> As I understand the current setup (which I never saw described anywhere
> in public, btw, till it came up on IRC today), we're importing trunk
> without history into an hg branch (call it "cvs-trunk").  Then we're
> merging trunk changes from CVS to cvs-trunk every 6 hours, with loss of
> history in the process (that is, all the changes in those six hours go
> into hg as a single commit). The mozilla2 work is happening on some
> other hg branch (call it "hg-trunk" for purposes of this discussion),
> with periodic manual merges from the cvs-trunk branch to hg-trunk (not
> sure whether this is a lossy process or not; someone more familiar with
> hg would have to comment).
>
> This is fine as a temporary setup, but I'm hoping there is a better plan
> for long-term, especially once all work is happening in hg.
>
> To be precise, my use case is needing to find out when a line of code
> was added and why.  I need to do this 5-10 times on a typical day.  If
> we simply take what's currently set up as hg-trunk and make it the
> trunk, with no other changes, this will mean that finding this
> information would involve both source repositories, looking at CVS log
> date ranges to find the possibly right checkins, etc.  This is a
> significant burden over the current "cvs annotate and search" workflow.
>
> I realize that we could possibly hack bonsai to do this behind the
> scenes, but I find myself working offline rather often now that I have a
> laptop.  I'm happy enough to cart around a local repository to handle
> history stuff, but carting around two of them, and manually moving from
> one to the other all the time, is a pain.
>
> So I'm hoping there is a plan to deal with this problem.  What is this
> plan?
>
> -Boris
>
> P.S. I understand there is a feeling in certain quarters that we do not
> in fact want "old" source history in the "mozilla2" repository.  I can't
> describe how strongly I object to that, for "It'd make working with the
> thing hellish" reasons.


_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

jonsmirl@gmail.com
In reply to this post by Johnny Stenback-3
On 4/26/07, Johnny Stenback <[hidden email]> wrote:

> FWIW, I can live w/o bringing in all the history prior to the point
> where we decide to start the Mozilla 2 repository, but I agree that
> loosing history in the import of changes from CVS from then on would
> really suck, a lot.
>
> Because of that, and the fact that there doesn't seem to be a solution
> out there that fits our needs, I've been hacking on a script that can
> import partial CVS history into a DVCS, producing as sane change sets as
> possible etc in the process. It's not beautiful or anything, but it
> appears to work pretty good so far. Should be ready for some real
> testing in a day or two.

Check out the source for cvsps, it can do most of what you need.

> Boris Zbarsky wrote:
> > I'm trying to understand the situation with source history in mozilla2.
> >
> > As I understand the current setup (which I never saw described anywhere
> > in public, btw, till it came up on IRC today), we're importing trunk
> > without history into an hg branch (call it "cvs-trunk").  Then we're
> > merging trunk changes from CVS to cvs-trunk every 6 hours, with loss of
> > history in the process (that is, all the changes in those six hours go
> > into hg as a single commit). The mozilla2 work is happening on some
> > other hg branch (call it "hg-trunk" for purposes of this discussion),
> > with periodic manual merges from the cvs-trunk branch to hg-trunk (not
> > sure whether this is a lossy process or not; someone more familiar with
> > hg would have to comment).
> >
> > This is fine as a temporary setup, but I'm hoping there is a better plan
> > for long-term, especially once all work is happening in hg.
> >
> > To be precise, my use case is needing to find out when a line of code
> > was added and why.  I need to do this 5-10 times on a typical day.  If
> > we simply take what's currently set up as hg-trunk and make it the
> > trunk, with no other changes, this will mean that finding this
> > information would involve both source repositories, looking at CVS log
> > date ranges to find the possibly right checkins, etc.  This is a
> > significant burden over the current "cvs annotate and search" workflow.
> >
> > I realize that we could possibly hack bonsai to do this behind the
> > scenes, but I find myself working offline rather often now that I have a
> > laptop.  I'm happy enough to cart around a local repository to handle
> > history stuff, but carting around two of them, and manually moving from
> > one to the other all the time, is a pain.
> >
> > So I'm hoping there is a plan to deal with this problem.  What is this
> > plan?
> >
> > -Boris
> >
> > P.S. I understand there is a feeling in certain quarters that we do not
> > in fact want "old" source history in the "mozilla2" repository.  I can't
> > describe how strongly I object to that, for "It'd make working with the
> > thing hellish" reasons.
>
>
> _______________________________________________
> dev-planning mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-planning
>


--
Jon Smirl
[hidden email]
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Benjamin Smedberg
In reply to this post by Johnny Stenback-3
Jon Smirl wrote:

> Check out the source for cvsps, it can do most of what you need.

cvsps has issues representing parts of our CVS tree, and the tool which does
imports from cvsps -> hg is even less capable [1].

I had decent luck with a tool that read the RCS files directly (cvs20hg),
but was unable to reimport after the initial import. The initial import took
1.5 weeks, which didn't inspire confidence about subsequent reimports being
made on a daily basis.

I don't know what jst's tool does, but I think our best bet would probably
be a post-commit hook in CVS which performs an equivalent hg commit.

--BDS

[1] http://benjamin.smedbergs.us/blog/2007-01-26/vcs-migration-headaches/
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

jonsmirl@gmail.com
On 4/26/07, Benjamin Smedberg <[hidden email]> wrote:
> Jon Smirl wrote:
>
> > Check out the source for cvsps, it can do most of what you need.
>
> cvsps has issues representing parts of our CVS tree, and the tool which does

I know for sure that cvsps can't import your whole repository. But
cvsps also has an incremental mode. In the incremental mode it
remembers the last revision number for each ,v file and only exports
the new stuff. I believe cvsps will work for doing the incremental
fetch.  The problems cvsps is encountering in the Mozilla repo are in
the older parts of the repo.

Of course you will need to manually create the file specifying the
versions numbers the first time. That file is normally created by
doing a whole repository import, and that doesn't work.

git has a mode where it monitors a cvs and automatically mirrors it
into a git tree. I believe they built it using the cvsps code as a
base.


> imports from cvsps -> hg is even less capable [1].
>
> I had decent luck with a tool that read the RCS files directly (cvs20hg),
> but was unable to reimport after the initial import. The initial import took
> 1.5 weeks, which didn't inspire confidence about subsequent reimports being
> made on a daily basis.

I was able to import the entire Mozilla repo with all of it's history
into git in four hours. I used my 2.8Ghz P4 desktop machine. I did
need to buy some more RAM (2.5GB). Have you considered using some of
the Mozilla resources to work on the Windows version of git?

The import was done using a hacked cvs2svn and git-fastimport.
cvs2svn has changed a lot but it would only take a day or so to hack
it back into supporting git-fastimport. The basic idea is that cvs2svm
converts the cvs repo into a generic change set representation which
gets piped to a specific backend like git or svn.

The main reason git-fastimport is fast is because it is a bulk
importer, it first imports everything and then indexes it all at once.
The other systems use a normal commit cycle for import which rebuilds
the indexes after each commit. That turns into an n-squared problem
and takes 1.5 weeks to run on a repo with 1 million change sets.
o(1 million) vs o(1 million * 1 million / 2)

>
> I don't know what jst's tool does, but I think our best bet would probably
> be a post-commit hook in CVS which performs an equivalent hg commit.
>
> --BDS
>
> [1] http://benjamin.smedbergs.us/blog/2007-01-26/vcs-migration-headaches/
> _______________________________________________
> dev-planning mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-planning
>


--
Jon Smirl
[hidden email]
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Robert Kaiser
In reply to this post by Benjamin Smedberg
Jon Smirl schrieb:
> I was able to import the entire Mozilla repo with all of it's history
> into git in four hours. I used my 2.8Ghz P4 desktop machine. I did
> need to buy some more RAM (2.5GB). Have you considered using some of
> the Mozilla resources to work on the Windows version of git?

Heh, and I thought I was the only one who felt that git was an
alternative to still consider ;-)

It looks like the MSYS port of git is progressing nicely, but still has
a few things to be done, mainly so that precompiled versions can be made
available. But I read this on their mailing list: "I can't say that the
port is fragile; I use it daily in a production environment" (Johannes Sixt)
I'd still think it would be cool if we'd look into that, but personally
I don't count on it any more, as the people who make those decision for
Mozilla seem to have their mind set on using hg.

Robert Kaiser
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Johnny Stenback-3
In reply to this post by Benjamin Smedberg
Benjamin Smedberg wrote:

> Jon Smirl wrote:
>
>> Check out the source for cvsps, it can do most of what you need.
>
> cvsps has issues representing parts of our CVS tree, and the tool which does
> imports from cvsps -> hg is even less capable [1].
>
> I had decent luck with a tool that read the RCS files directly (cvs20hg),
> but was unable to reimport after the initial import. The initial import took
> 1.5 weeks, which didn't inspire confidence about subsequent reimports being
> made on a daily basis.
>
> I don't know what jst's tool does, but I think our best bet would probably
> be a post-commit hook in CVS which performs an equivalent hg commit.

Here's an outline of what the tool does. The tool does *not* use any CVS
hooks or anything like that. It's way more straight forward. The is
based on finding differences between two CVS trees and using that to
figure out what to bring over to hg (or whatever VCS for that matter).
So the way we'd use this tools (initially) is basically to:

- Check out our base tag (MOZILLA_1_9a3_RELEASE or whatever base we
pick), that'll be our source tree.
- Copy the source tree to a destination tree.
- Commit the destination tree into hg (ignoring .cvsignore and CVS dirs
etc).
- Update to the CVS trunk in the source tree (trying to get a sane state
etc).
- Run the tool, pointing it at the source tree and the destination tree.

The tool then goes through the two trees and finds all the files listed
in the CVS/Entries files and remembers their versions. Once it has all
that info it walks through that info and finds the differences, i.e.
changed files, new files, deleted files. Then it goes through the
changed files and uses cvs log to find the checkin comments etc for all
the versions for all changed files between the source and the
destination. Once it has that information it walks through all that and
combines the checkins that are by the same author and the same checkin
message and are within 60 seconds of each other, and calls that a change
set. Then it goes through those change sets (sorted by checkin time),
updates each file in the change set in the destination tree to the
version in the change set, adds/removes added/removed files from hg, and
checks in the changes into hg.

Once that's all done we can diff the tree for sanity checking etc.

Then we wait some number of hours and update the source tree again, run
the tool, and repeat.

And in the event that something would go horribly wrong, we can always
roll back to a previous known good state and start over from there.

This tool has already ran through all changes in CVS from April 19th
through yesterday. For testing purposes this is all in a git repository
now and I'm happy to share that if anyone thinks that's of any value.

The tool is available here:

http://dp.jstenback.com/git/?p=hg-track-cvs;a=blob;hb=HEAD;f=hg-track-cvs.pl

All the logic around running the tool still needs done, and there's a
bunch of ways we can do that and I'm happy to discuss ideas on that. But
the hard part is done and seems to be working flawlessly so far. Once
the tool is used only to bring in changes that have happened in the last
N hours where N is less than a day or whatever, the tool runs in a
matter of a few minutes (depending on how hot the disk cache is etc), so
we could even run it every hour or so, making sure we don't get
mid-checkins etc.

And yes, I'm happy to maintain this tool, there's really nothing
complicated about it and there's a bunch of commented out prints etc to
aide in debugging if anything does go wrong.

>
> --BDS
>
> [1] http://benjamin.smedbergs.us/blog/2007-01-26/vcs-migration-headaches/


_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Graydon Hoare-3
Johnny Stenback wrote:

> All the logic around running the tool still needs done, and there's a
> bunch of ways we can do that and I'm happy to discuss ideas on that. But
> the hard part is done and seems to be working flawlessly so far. Once
> the tool is used only to bring in changes that have happened in the last
> N hours where N is less than a day or whatever, the tool runs in a
> matter of a few minutes (depending on how hot the disk cache is etc), so
> we could even run it every hour or so, making sure we don't get
> mid-checkins etc.

I should have clarified in my earlier post today about technical
feasibility: forming tree-wide commit history for one lineage of a
collection of ,v files -- say their shared trunk -- is relatively
doable. It's still possible to have time-interleaved commits to
different parts of a tree, but there's at least *some* linearity and
some sensible strategies for tie-breaking the interleaved cases. This is
what jst's tool does, and things like cvsps, svn2cvs, the mtn importer,
etc. can do so as well.

The "research grade" difficulty I alluded to comes with symbols and
branching: the files making up a CVS archive can have mutually
incompatible notions of branch points, branch membership, and tag names,
such that there is no acceptable topology for the output graph that fits
all the input files. Reconciling those -- and remarkably, real
repositories often contain such cases -- is the hard heuristic problem.

IOW, if you're willing to forget about other branches, the history
problem becomes a fair bit easier.

-graydon
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Boris Zbarsky
Graydon Hoare wrote:
> The "research grade" difficulty I alluded to comes with symbols and
> branching

Ah, good point.

For our existing release branches (which don't merge back into the trunk), I
think I would be happy enough with having to use the CVS repository to get
change history (especially because this is a much rarer occurrence).  I suspect
it would also be possible to modify bonsai to do this transparently.

For feature branches that merged back to the trunk and then landed as a flat
commit life already sucks to some extent.  Enough so that again I would be
willing to deal with having to go to the CVS repository in the cases when a
change comes from one of those branches...  I'm not sure that we could even make
bonsai deal with this gracefully, but that can be sorted out in the future.

> IOW, if you're willing to forget about other branches, the history
> problem becomes a fair bit easier.

I think I'm willing to forget about other branches, especially if my other
alternative is a flat initial commit of the current repository snapshot...

That said, the question of how taking that approach would affect the feasibility
of your flag day scenarios remains.  Would we be burning any bridges by doing
this that we wouldn't burn with a flat-commit approach?

-Boris

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Jonathan Watt-3
Boris Zbarsky wrote:
>> IOW, if you're willing to forget about other branches, the history
>> problem becomes a fair bit easier.
>
> I think I'm willing to forget about other branches, especially if my
> other alternative is a flat initial commit of the current repository
> snapshot...

Since I previously made unhappy noises about not having pre-mozilla2 history in
hg I'd better clarify that for me too it's only trunk that's of real concern.
I'm not often interested in history on branches and I'm happy to use CVS when I am.

Jonathan
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Jonas Sicking-2
In reply to this post by Graydon Hoare-3
Graydon Hoare wrote:
> IOW, if you're willing to forget about other branches, the history
> problem becomes a fair bit easier.

I know one argument against importing too much history into hg has been
that with hg and many other DVCSs the user downloads the entire history
database locally, which for a tree of mozillas size and age is a
considerable amount of data.

But if we stick to just importing the trunk, maybe that will be less
enough data that we might be able to import a longer time back.
Personally I every so often dig back many years into history to find out
why a change was made, so the longer history of trunk we can import the
better.

/ Jonas
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Brendan Eich-2
On May 1, 4:08 pm, Jonas Sicking <[hidden email]> wrote:
> But if we stick to just importing the trunk, maybe that will be less
> enough data that we might be able to import a longer time back.
> Personally I every so often dig back many years into history to find out
> why a change was made, so the longer history of trunk we can import the
> better.

I do cvs archeology a lot too. I'm still going to have a cvs working
tree from the trunk (so are you). I'm happy to use that and not burden
everyone who ever pulls from the new repository with the full trunk
history. I believe most hackers in three years will not be doing
intensive archeology back to the pre-hg days.

But let's say we exclude branches. I recall even with just trunk, the
initial pull was 1GB (please correct me if I'm misremembering). If so,
that seems too big to me.

/be

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Boris Zbarsky
[hidden email] wrote:
> But let's say we exclude branches. I recall even with just trunk, the
> initial pull was 1GB (please correct me if I'm misremembering). If so,
> that seems too big to me.

Just the tip snapshot (no history) is around 500MB.

So 1GB for that plus trunk history (including commit comments!) is not
particularly unlikely, yes...  Where between 500MB and 1GB is the line where you
become unhappy?  ;)  Frankly, I'm about equally unhappy about both numbers.

That said, how well does it compress?  The tip snapshot compresses to 35MB, right?

-Boris

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

L. David Baron
In reply to this post by Boris Zbarsky
On Tuesday 2007-05-01 17:47 -0500, Boris Zbarsky wrote:
> > IOW, if you're willing to forget about other branches, the history
> > problem becomes a fair bit easier.
>
> I think I'm willing to forget about other branches, especially if my other

I'd be quite happy to have a trunk-only import as well.  (And I was
actually going to respond to Graydon's previous message questioning
whether the hard problems applied to trunk-only imports, but hadn't
gotten around to it yet.)

It's also worth noting that we can always import release branches
later (since they don't merge back to the trunk).  We could even
later import topic branches, just without the final merge.

-David

--
L. David Baron                                <URL: http://dbaron.org/ >
           Technical Lead, Layout & CSS, Mozilla Corporation

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Axel Hecht
In reply to this post by Boris Zbarsky
I have an only slightly off topic question, when we're talking about
loosing branches, how does that look going forward?

I'm particularly interested in having support for bonsai revision
graphs, I'm using those on at least a weekly basis for finding out which
changes made it into which release. I recall svn having issues there,
how are our other combatants doing?

Axel
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Benjamin Smedberg
In reply to this post by Brendan Eich-2
[hidden email] wrote:

> I do cvs archeology a lot too. I'm still going to have a cvs working
> tree from the trunk (so are you). I'm happy to use that and not burden
> everyone who ever pulls from the new repository with the full trunk
> history. I believe most hackers in three years will not be doing
> intensive archeology back to the pre-hg days.

I hope to moot this discussion by using cvs20hg, but...

1) Disjunct blame is not useful. Disjunct VCS log is fairly workable,
because you have a singular place to switch from one system to the other.
But blame is in general much more useful than log, and using blame across
multiple VCS boundaries requires matching the affected lines across the gap,
which is *really hard*.

2) mozilla2 cannot rewrite all our code to the extent that hackers in three
years won't find blame crucially heopful. Even if automated tools rewrite
half the lines in our codebase over three years, the actual bugfixes and
edge cases of our current codebase are still going to exist and be a
valuable reference.

That being said, I don't use CVS log or blame directly, and I can't imagine
most other people do either. Bonsai is our pearl of great price here, and if
we could make bonsai do the right thing with the import boundary that would
solve all the issues I have.

This is not an argument for stopping activity now. But saying that complete
VCS archaeology is not a desirable goal is a huge mistake.

--BDS
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Benjamin Smedberg
In reply to this post by Brendan Eich-2
[hidden email] wrote:

> But let's say we exclude branches. I recall even with just trunk, the
> initial pull was 1GB (please correct me if I'm misremembering). If so,
> that seems too big to me.

Complete history of all files (which is much more than we actually import
for mozilla2) is 1.3G. I don't think this is excessive at all, except for
getting it the first time. Compressed, the full repository is 859MB.

--BDS
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: source history in mozilla2

Robert Kaiser
Benjamin Smedberg schrieb:
> Complete history of all files (which is much more than we actually import
> for mozilla2) is 1.3G. I don't think this is excessive at all, except for
> getting it the first time. Compressed, the full repository is 859MB.

For what is it, i.e. complete history of all files (full repository or
"just" what FF uses?), that actually sounds not big to me... Even more
interesting is what the mozilla2 subset is in a repo, esp. in a
comparison of full trunk history vs. single snapshot import - as that is
the real difference that decision makes.

Robert Kaiser
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
12