Question about your huge work

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about your huge work

tanstaafl-2
Hello Ishikawa,

I'm very curious to hear just how close you are to getting all of your
extremely hard work on fixing many of the network:protocol bugs that you
have been working on for - what, years now?

I am very much looking forward to seeing these integrated, especially if
they help the IMAP communication processes.

I was of course disappointed it didn't make it into 68, but I know it is
a lot of work (I've been watching your progress on the list), so am very
hopeful - do you think it will make it into the next major release (78?)?

Thanks very much again for your work on this!

Charles
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Question about your huge work

ISHIKAWA,chiaki
On 2020/02/13 0:30, Tanstaafl wrote:

> Hello Ishikawa,
>
> I'm very curious to hear just how close you are to getting all of your
> extremely hard work on fixing many of the network:protocol bugs that you
> have been working on for - what, years now?
>
> I am very much looking forward to seeing these integrated, especially if
> they help the IMAP communication processes.
>
> I was of course disappointed it didn't make it into 68, but I know it is
> a lot of work (I've been watching your progress on the list), so am very
> hopeful - do you think it will make it into the next major release (78?)?
>
> Thanks very much again for your work on this!
>
> Charles
>

Dear Charles,

Thank you for your kind words.

I have not given up. :-)

The patch may not immediately IMAP communication processes, but
it probably flags many I/O errors during runtime that will help
developers to figure out how to fix problematic code pieces over the
long term.

I am hoping to tidy up my patches using more MACROs to reduce the
disruptive source changes so that the number of source lines inserted
would be smaller.

The issue here is that there *ARE* so many places where the low-level
I/O routine, especially, output side is NOT checked for error return
value, and I need to insert code to flag and report the errors somehow
to the users and developers. The unchecked parts are simply so many so I
have to touch many lines, and initially I simply inserted printf-style
dump statements. As a hindsight and was pointed out by many including
Jorg, I agree the dump statements should be reduced, and I think I can
do this at least superficially by hiding the verbose output inside MACROs.

Also, I wanted to make sure that the rather large surgical changes won't
affect correct operation of TB in C-C source tree. I want to make sure
that the tests available will pass. This has been a up-hill battle for a
large patch set outside the official tree.
(It took me several months to salvage my patches to match the large
source tree change in the last 18 months, for example.)
But finally, I could make the patch work again last October (correct
|make mozmill| and more or less acceptable xpcshell-test passage.).

A couple of things slowed down my process since December.

1. mozmill to mochitest transition.

|make mozmill| test which I have used for many years has been replaced
by mochitest.The transition took place in December.
It took me a couple of months to make sure it works well on my local
linux machine.: That I use GCC-9 instead of clang locally uncovered a
few issues of default standard test framework.
It now runs locally. I am in the process of modifying shell/awk scripts
I used to extract information from mozmill test log.
The scripts shall collect and sort the errors/warnings in the order
frequencies, and flag hither-to-unknown errors, etc.
There *ARE* suspicious errors, but these seem to have existed BEFORE my
patches... :-(
They are disturbing, but they are not caused by my patches.

2. xpcshell-tests issues.

Also, in parallel, I really wanted to make sure that my changes won't
affect xpcshell-test. Superficially, it doesn't. To be honest, there
have been errors which seem to occur only on my local PC and not
tryserver. Strange.

However, I felt very uncomfortable. The reason that TB with my patches
may run well without hidden problems could be that xpcshell-test does
NOT print out logs (including warnings/errors UNLESS each unit fails.)
So we are shielded from many such suspcious errors and warnings most of
the time.
However, mozilla test suite is known to LET A TEST SUCCEED even if there
*ARE* underyling SERIOUS ERRORS.
So I have enabled the --verbose flags to find so many warnings/errors,
and was overwhelmed at the amount of such log lines.
For now, I simply ignore many of them, and decided to run xpcshell-tests
under valgrind to make sure that my patches won't cause memory-related
problems.

I used to run mozmill tests under valgrind.: it took more than a day for
the whole test run to finish, but I could find several bugs related to
memory allocation this way over the years.
So I wanted to make sure that xpcshell-test is free from memory problems
with my patches.

Now, to my utter surprise, there *ARE* uninitialized memory access
errors during xpcshell-tests even without my patches(!). I am glad that
I ran xpcshell-test under valgrind. (It took them 13 hours or so to
finish. A few tests do not run due to timeout. But that is life.
All of them seem to be benign (may not be optimal in terms of
performance/expectation), but I had to fix them one way or the other
because I want TB to be a rock-solid mail client (tm).
Well (tm) is just a joke.

The above two factors that sort of side-tracked me for the last couple
of months have been taken care of now. Actually, I have not reported (I
realized now) another valgrind memory error which has been fixed
locally. But it is not a big issue now that the fix is in sight.

So, I intend to clean up my I/O patches using more MACROS hopefully in
the next several weeks.

It is just that many organizations in Japan finish their fiscal year
accounting at the end of March including my office. (Government offices
end the fiscal year at the end of March, and schools also.)
So my day job has a lot of excess work until the end of March.
Actually, December is a big technology exhibition time for the last 15+
years and I could not do a lot from November to March time frame. :-(

So I may be slow moving until the end-of-March workload shrinks.
But I will try.


BTW, my given name is Chiaki. You can call me Chiaki.
I should probably write my e-mail address as
"Chiaki Ishikawa" < ishikawa.yk.rim.or.jp>
A complicating factor is, in Japan, we use the family name first.
I thought "ISHIKAWA,Chiaki" might indicate the original Japanese order.

I know people in Hungary also use the family name first and one
Hungarian American person I know used
FAMILYNAME, Givenname
on his name card and I adopted the order/style somehow.
But I found that this may not translate well after all. :-)

Thank you again.

Chiaki
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird