C-C TB Try server issue: Windows build fails most of the time for me.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

C-C TB Try server issue: Windows build fails most of the time for me.

ISHIKAWA,chiaki
Hi,

For the last few weeks,
I get win32 build/configure error on the Tryserver when I submit a job
that includes the win32 debug build.

--- begin quote: excerpt of the error ---
mozmake.exe[1]: *** No rule to make target
'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/mozilla/aclocal.m4',
needed by
'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/configure'. Stop.
mozmake.exe: ***
[c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/objdir-tb/Makefile]
Error
--- end quote

What went wrong?

For example, the try job:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=7983cac21e61

Can anyone think of what I am doing wrong.
The used patches are in the above URL.
(notably I used client-py.patch to apply patches to ./mozilla
subdirectory,  and win32 hack to remove
ALWAYS_RUN_CLIENT_PY=1, etc.)

I don't know any other requirement specific to windows build.

Linux build works OK.

MacOS X 10.x occasionally barfs due to some esoteric system-dependent
issue or configuration issue, but more or less it is OK since I can
guess why the build fails and remedy the situation.

It is windows build that has failed me.

A few times, the build worked for no apparent reason, well, at least not
clear why windows build worked on those particular submissions. That
confuses me to no end.

TIA
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: C-C TB Try server issue: Windows build fails most of the time for me.

Mike Hommey
On Wed, Jul 08, 2015 at 04:03:20AM +0900, ISHIKAWA, Chiaki wrote:

> Hi,
>
> For the last few weeks,
> I get win32 build/configure error on the Tryserver when I submit a job
> that includes the win32 debug build.
>
> --- begin quote: excerpt of the error ---
> mozmake.exe[1]: *** No rule to make target
> 'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/mozilla/aclocal.m4',
> needed by
> 'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/configure'. Stop.
> mozmake.exe: ***
> [c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/objdir-tb/Makefile]
> Error
> --- end quote
>
> What went wrong?
>
> For example, the try job:
> https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=7983cac21e61
>
> Can anyone think of what I am doing wrong.
> The used patches are in the above URL.
> (notably I used client-py.patch to apply patches to ./mozilla
> subdirectory,  and win32 hack to remove
> ALWAYS_RUN_CLIENT_PY=1, etc.)
>
> I don't know any other requirement specific to windows build.
>
> Linux build works OK.
>
> MacOS X 10.x occasionally barfs due to some esoteric system-dependent
> issue or configuration issue, but more or less it is OK since I can
> guess why the build fails and remedy the situation.
>
> It is windows build that has failed me.
>
> A few times, the build worked for no apparent reason, well, at least not
> clear why windows build worked on those particular submissions. That
> confuses me to no end.

Looks like your build didn't clone mozilla-central for whatever reason.

Mike
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: C-C TB Try server issue: Windows build fails most of the time for me.

ISHIKAWA,chiaki
In reply to this post by ISHIKAWA,chiaki
On 2015/07/08 6:32, Mike Hommey wrote:
 > On Wed, Jul 08, 2015 at 04:03:20AM +0900, ISHIKAWA, Chiaki wrote:>>
Hi,>>>> For the last few weeks,>> I get win32 build/configure error on
the Tryserver when I submit a job>> that includes the win32 debug
build.>>>> --- begin quote: excerpt of the error --->> mozmake.exe[1]:
*** No rule to make target>>
'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/mozilla/aclocal.m4',>>
needed by>>
'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/configure'.
Stop.>> mozmake.exe: ***>>
[c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/objdir-tb/Makefile]>>
Error>> --- end quote>>>> What went wrong?>>>> For example, the try
job:>>
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=7983cac21e61>>>>
Can anyone think of what I am doing wrong.>> The used patches are in the
above URL.>> (notably I used client-py.patch to apply patches to
./mozilla>> subdirectory,  and win32 hack to remove>>
ALWAYS_RUN_CLIENT_PY=1, etc.)>>>> I don't know any other requirement
specific to windows build.>>>> Linux build works OK.>>>> MacOS X 10.x
occasionally barfs due to some esoteric system-dependent>> issue or
configuration issue, but more or less it is OK since I can>> guess why
the build fails and remedy the situation.>>>> It is windows build that
has failed me.>>>> A few times, the build worked for no apparent reason,
well, at least not>> clear why windows build worked on those particular
submissions. That>> confuses me to no end.> > Looks like your build
didn't clone mozilla-central for whatever reason.> > Mike>
You mean that "for windows build", my build didn't clone mozilla-central (?)

[linux and macos X build works, so they seem to clone mozilla-central
correctly. Correct? Or did my job fail to clone it? ]

Then I wonder if the unwanted behavior can be
related to the following issue

Bug 1154222 - comm-central tryserver does not clean mozilla patches if
m-c tip has not changed

mentioned in the "Known issues" section of
https://wiki.mozilla.org/ReleaseEngineering/ThunderbirdTryServer

But it may not explain the random nature of the failure and the above
bug seems to apply linux and macos X build as well.
(Oh, is my trying to mix windows build with linux and macos X builds
triggering something on the tryserver?)

Anyway, I will submit the job again to see if anything changes...

TIA

_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: C-C TB Try server issue: Windows build fails most of the time for me.

ISHIKAWA,chiaki
In reply to this post by ISHIKAWA,chiaki
(Sorry for top-posting).

Oops, sorry the "Bug 1143570 - Copy/Paste into plain text editor deletes
newlines from quoted text" seems to hit this.
I didn't realize the loss of newlines while I was editing the e-mail.

Yes, I use TB for all my e-mail needs these days in a cross-platform manner
(mostly linux and windows).

Now I realize if I ask for windows 64-bit debug build on try server,
I simply get windows 32-bit build (and testing, etc. will be done using
win32 binary),
and so I can focus why win32 build fails most of the time for my job submission.

- Plan 1. I will compare how other people's win32 build is handled (already
I notice there are jobs that do not seem to have win32 specific tryserver
job submission patches. I have no idea why they seem to work.), and

- PLan 2. I decide to check M-C changes by submitting firefox tryserver job,
assuming
     windows build for ff tryserver works for me. (I think the chance is
higher. C-C TB build
     on tryserver, especially windows, is getting very complex.)

Plan 2 is not ideal for my purpose of reporting and fixing bug in win32
version of C-C TB, but
getting the patch accepted into M-C trunk is more important (then the source
change will be effective on C-C TB side eventually) and
I can usually test the patch locally very well for linux and usually the
bug/fix is platform-agnostics and this should go a long way for fixing
noticed issues in TB.

TIA

CI

On 2015年07月08日 11:37, ISHIKAWA, Chiaki wrote:

> On 2015/07/08 6:32, Mike Hommey wrote:
>> On Wed, Jul 08, 2015 at 04:03:20AM +0900, ISHIKAWA, Chiaki wrote:>> Hi,>>>> For the last few weeks,>> I get win32 build/configure error on the
> Tryserver when I submit a job>> that includes the win32 debug build.>>>> ---
> begin quote: excerpt of the error --->> mozmake.exe[1]: *** No rule to make
> target>>
> 'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/mozilla/aclocal.m4',>>
> needed by>>
> 'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/configure'.
> Stop.>> mozmake.exe: ***>>
> [c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/objdir-tb/Makefile]>>
> Error>> --- end quote>>>> What went wrong?>>>> For example, the try job:>>
> https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=7983cac21e61>>>>
> Can anyone think of what I am doing wrong.>> The used patches are in the
> above URL.>> (notably I used client-py.patch to apply patches to ./mozilla>>
> subdirectory,  and win32 hack to remove>> ALWAYS_RUN_CLIENT_PY=1, etc.)>>>>
> I don't know any other requirement specific to windows build.>>>> Linux
> build works OK.>>>> MacOS X 10.x occasionally barfs due to some esoteric
> system-dependent>> issue or configuration issue, but more or less it is OK
> since I can>> guess why the build fails and remedy the situation.>>>> It is
> windows build that has failed me.>>>> A few times, the build worked for no
> apparent reason, well, at least not>> clear why windows build worked on
> those particular submissions. That>> confuses me to no end.> > Looks like
> your build didn't clone mozilla-central for whatever reason.> > Mike>
> You mean that "for windows build", my build didn't clone mozilla-central (?)
>
> [linux and macos X build works, so they seem to clone mozilla-central
> correctly. Correct? Or did my job fail to clone it? ]
>
> Then I wonder if the unwanted behavior can be
> related to the following issue
>
> Bug 1154222 - comm-central tryserver does not clean mozilla patches if m-c
> tip has not changed
>
> mentioned in the "Known issues" section of
> https://wiki.mozilla.org/ReleaseEngineering/ThunderbirdTryServer
>
> But it may not explain the random nature of the failure and the above bug
> seems to apply linux and macos X build as well.
> (Oh, is my trying to mix windows build with linux and macos X builds
> triggering something on the tryserver?)
>
> Anyway, I will submit the job again to see if anything changes...
>
> TIA
>
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: C-C TB Try server issue: Windows build fails most of the time for me.

ISHIKAWA,chiaki
In reply to this post by ISHIKAWA,chiaki
On 2015/07/09 16:50, ishikawa wrote:

> On 2015年07月09日 13:16, Edmund Wong wrote:
>> Mike Hommey wrote:
>>> On Wed, Jul 08, 2015 at 04:03:20AM +0900, ISHIKAWA, Chiaki wrote:
>>>> Hi,
>>>>
>>>> For the last few weeks,
>>>> I get win32 build/configure error on the Tryserver when I submit a job
>>>> that includes the win32 debug build.
>>>>
>>>> --- begin quote: excerpt of the error ---
>>>> mozmake.exe[1]: *** No rule to make target
>>>> 'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/mozilla/aclocal.m4',
>>>>
>>>> needed by
>>>> 'c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/configure'. Stop.
>>>> mozmake.exe: ***
>>>> [c:/builds/moz2_slave/tb-try-c-cen-w32-d-00000000000/build/objdir-tb/Makefile]
>>>>
>>>> Error
>>>> --- end quote
>>>>
>>>> What went wrong?
>>>>
>>>> For example, the try job:
>>>> https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=7983cac21e61
>>>>
>>
>> The problem from what I see is that you removed the client.py running
>> via your mozconfig changes:
>>
>>   # Run client.py
>>      1.11  mk_add_options CLIENT_PY_ARGS="$([ -f
>> $topsrcdir/build/client.py-args ] && cat $topsrcdir/build/client.py-args)"
>>      1.12 -mk_add_options ALWAYS_RUN_CLIENT_PY=1
>>
>> TB's buildbot releng (from what I see) requires that to be run otherwise
>> it doesn't get the m-c code.
>>
>>
>> Edmund
>
> I see, thank you for noticing the problem.
>
> Short Answer: I think I don't need this removal any more.
>
> Long answer:
>
> I wondered why I did this and vaguely recall I had some Q&A sessions on the
> issue.
>
> Searching via google, I realized that, back in 2013, I was advised to delete
> "ALWAYS_RUN_CLIENT_PY" from win32 client.py.
> Reason back then was that the tryserver tried to apply patches twice and
> failed for me.
>
> http://comments.gmane.org/gmane.comp.mozilla.devel.platform/2920
>
> Also from the search, I noticed in
> https://wiki.mozilla.org/Thunderbird/StatusMeetings/2011-06-21
>
> --- begin quote
> Infrastructure Update
>
>      bug 656049 c-c:0f79491664e9 client.mk can now invoke client.py for
> comm-central builds via .mozconfig
>
>   mk_add_options CLIENT_PY_ARGS="--verbose --skip-venkman"
>   mk_add_options ALWAYS_RUN_CLIENT_PY=1
>
> This can be directly invoked as make checkout, or implicitely whenever using
> make some-target when ALWAYS_RUN_CLIENT_PY is set.
>
> Coming to Try Server soon...
> --- end quote
>
> Maybe either I misread the tips, and was too eager to remove this
> "ALWAYS_RUN_CLIENT_PY" in not-to-be-touched files [I thought it was likely,
> but looking at the patch I am not so sure now.]
> or maybe with the changes in between 2013 and 2015, this removal
> "ALWAYS_RUN_CLIENT_PY" may not be necessary any more. (I think this is the
> case now.)
>
> All I can say is that there seemed to be an interaction between
> --apply-patches and
> ALWAYS_RUN_CLIENT_PY in unwanted manner back in 2013.
>
> As I wrote the post-postscript re my observation testing framework failure,
> I realize that
> the recent post to explain the sorting order issue of handling M-C patches
> by Joshua Cranmer
> solves this question for me. His script no longer removes the
> "ALAYWAS_RUN_CLIENT_PY" line!
> I should have read the script carefully. There was "#" before the sed
> command to do exactly that! But back then, I was more eager to figure out
> the filename sorting issue.)
>
> Thank you again for spotting this problem. I could not figure this out myself.
>
> CI
>
> PS: In the meantime, I tried FF build with the M-C patches I wanted to test,
> and
> it compiled, *BUT* the test seems broken.
>
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=d16bb4c8debf
>
> I mean the testing framework hiccuped.
> It said "success" for win32  xpcshell tests (and later for win8 test, too),
> and I could not believe it since xpcshell-tests on C-C TB side is full of
> warnings and intermittent errors.
>
> When I checked the error log, I saw very short log.
> I noticed the testing frameworks barfed and died prematurely without running
> any tests, but it was reported as "success".
>
> Ah, testing is fun :-(
>
> At least, I could verify that the my updated patches compile under windows :-).
>
> PPS:
>
> win32 test barfed due to WindowsError: [Error 6] The handle is invalid.
> win8 version seems to have failed due to " 23:43:00 INFO - WindowsError:
> [Error 5] Access is denied".
>
> I wonder if the test frame work's access issue is related to "There is a bug
> in that, under certain circumstances, it could try to apply patches due to
> working directory reuse on build slaves." mentioned in the post to my
> question last month by Joshua Cranmer.
>
> https://groups.google.com/forum/#!topic/mozilla.dev.builds/pRr8gw0Ued0
>
> Hist post is in the latter half of the post, and while I was re-reading his
> post, I realize
> there is an answer to "ALAYWAS_RUN_CLIENT_PY" (!).
> Between 2013-2015, there was a change to make the deletion of the macro
> unnecessary !
>

By not removing "ALWAYS_RUN_CLIENT_PY", my submission finally succeeded
in getting windows version compiled (!)
Come to think of it, the first few times windows compilation succeeded
was when I failed to apply this unnecessary patch :-(
I tweaked my local patch preparation script to make sure this
extra patch is not included, etc.

So it is confirmed that ALWAYS_RUN_CLIENT_PY should not be removed any
more (unlike the time when I had an issue back in 2013.)

I would like to thank everyone who took the time to figure out my problem!

CI


_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird