comments on xxxUtf16() I/O calls

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

comments on xxxUtf16() I/O calls

Aleksey Sanin-2
Apologies for the rant and bringing back this old topic but ...

 From NSPR user point of view I have to say that the new set
of xxxUtf16() I/O calls is really horrible. These new calls
are implemented on Windows only and in order to use them (and
get correct I18N on Windows), the developer has to create wrappers
for *every* NSPR I/O call (call xxxUtf16() function on Windows,
call regular function on other platforms).

To large degree, this kills the whole point for having NSPR
as a cross-platform layer. Worse, two sets of APIs makes it
very hard to do "the right thing". It is very easy for developers
to forget about these problems and use incorrect function.
Later it is very hard to track down these kind of mistakes and
they tend to show up in the worst possible time.

Linux/*BSD I/O calls for the long time accept UTF8 filenames and
NSPR works great with UTF8 strings on these operating systems.
IMHO, it is much easier to just agree that on all platforms all
NSPR functions just accept UTF8 and then do internal conversion
if needed.

I am not very familiar with browser source code but I guess there
were reasons for creating this second API... It is just sad that
it was done this way :(


Aleksey
_______________________________________________
dev-tech-nspr mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-nspr
Reply | Threaded
Open this post in threaded view
|

Re: comments on xxxUtf16() I/O calls

Darin Fisher-2
Thanks for the feedback.  The UTF-16 APIs are not part of the official
NSPR release.  They were added as a trial API and are not compiled by
default.  They have never actually been used by any Mozilla browser
releases.

Personally, I think it makes sense to provide both narrow and wide
APIs on all platforms.  If your application wishes to optimize for
Windows, then you can call the UTF-16 set, but if you don't care about
Windows, then the standard NSPR APIs may be more suitable.

-Darin



On 5/31/06, Aleksey Sanin <[hidden email]> wrote:

> Apologies for the rant and bringing back this old topic but ...
>
>  From NSPR user point of view I have to say that the new set
> of xxxUtf16() I/O calls is really horrible. These new calls
> are implemented on Windows only and in order to use them (and
> get correct I18N on Windows), the developer has to create wrappers
> for *every* NSPR I/O call (call xxxUtf16() function on Windows,
> call regular function on other platforms).
>
> To large degree, this kills the whole point for having NSPR
> as a cross-platform layer. Worse, two sets of APIs makes it
> very hard to do "the right thing". It is very easy for developers
> to forget about these problems and use incorrect function.
> Later it is very hard to track down these kind of mistakes and
> they tend to show up in the worst possible time.
>
> Linux/*BSD I/O calls for the long time accept UTF8 filenames and
> NSPR works great with UTF8 strings on these operating systems.
> IMHO, it is much easier to just agree that on all platforms all
> NSPR functions just accept UTF8 and then do internal conversion
> if needed.
>
> I am not very familiar with browser source code but I guess there
> were reasons for creating this second API... It is just sad that
> it was done this way :(
>
>
> Aleksey
> _______________________________________________
> dev-tech-nspr mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-nspr
>
_______________________________________________
dev-tech-nspr mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-nspr
Reply | Threaded
Open this post in threaded view
|

Re: comments on xxxUtf16() I/O calls

Aleksey Sanin-2
In reply to this post by Aleksey Sanin-2

> Personally, I think it makes sense to provide both narrow and wide
> APIs on all platforms.  If your application wishes to optimize for
> Windows, then you can call the UTF-16 set, but if you don't care about
> Windows, then the standard NSPR APIs may be more suitable.

Bug filed:

https://bugzilla.mozilla.org/show_bug.cgi?id=339880

I should have a patch that makes NSPR I/O calls "UTF8 ready" in the
next couple days and this patch will be attached to the bug above.

Aleksey
_______________________________________________
dev-tech-nspr mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-nspr
Reply | Threaded
Open this post in threaded view
|

Re: comments on xxxUtf16() I/O calls

Wan-Teh Chang
Aleksey Sanin wrote:

>
>> Personally, I think it makes sense to provide both narrow and wide
>> APIs on all platforms.  If your application wishes to optimize for
>> Windows, then you can call the UTF-16 set, but if you don't care about
>> Windows, then the standard NSPR APIs may be more suitable.
>
> Bug filed:
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=339880
>
> I should have a patch that makes NSPR I/O calls "UTF8 ready" in the
> next couple days and this patch will be attached to the bug above.

Aleksey,

Thanks for your opinion and patch.

Unfortunately we can't change the standard NSPR APIs
to use UTF-8 file pathnames.  We will need to add new
functions that interpret the "char *" pathname arguments
as UTF-8 strings.  As you pointed out in the bug report,
applications and libraries that use NSPR won't be able
to benefit from the new functions automatically.  But
we must ensure that the current functions continue to
interpret the "char *" pathname arguments as strings
in the "default" or "native" character encoding.

Since Mac OS X and Linux already use UTF-8 as the
default/native character encoding, I'm wondering if
it is possible to change the default/native code
page of a process or even a thread to CP_UTF8 on
Windows.  In other words, can we set CP_ACP or
CP_THREAD_ACP to CP_UTF8?  I just spent about 20
minutes researching this topic but didn't find anything.
Does anyone know if this is possible?

Wan-Teh

_______________________________________________
dev-tech-nspr mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-nspr
Reply | Threaded
Open this post in threaded view
|

Re: comments on xxxUtf16() I/O calls

Aleksey Sanin-2
In reply to this post by Aleksey Sanin-2
 > In other words, can we set CP_ACP or
> CP_THREAD_ACP to CP_UTF8?  I just spent about 20
> minutes researching this topic but didn't find anything.
> Does anyone know if this is possible?

setlocale() ?

Aleksey
_______________________________________________
dev-tech-nspr mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-nspr
Reply | Threaded
Open this post in threaded view
|

Re: comments on xxxUtf16() I/O calls

Aleksey Sanin-2
In reply to this post by Aleksey Sanin-2
 > As you pointed out in the bug report,
> applications and libraries that use NSPR won't be able
> to benefit from the new functions automatically.

Well, actually I said that some applications/libraries
will benefit and some will not. E.g. NSS_Init() function
works just fine with UTF8 strings after the NSPR upgrade
to handle UTF8 filenames :)

Again, it significantly depends on the application/library.
If the application/library was written with "current locale"
in mind then this application will be broken with proposed
UTF8 upgrade for NSS. But in reality, these applications/libraries
are already broken to some degree because they are not really
I18N ready (e.g. no Japanese filename with Chinese default
locale). Thus, these applications/libraries *have* to be
changed anyway...

BTW, don't take me wrong. I do understand all the issues
with ABI/API backward compatibility, etc. I just feel that
the current NSPR model for handling I18N issues is severely
broken. And adding another set of calls (Utf8 or Utf16 or ???)
will make things even worse.


Aleksey
_______________________________________________
dev-tech-nspr mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-nspr
Reply | Threaded
Open this post in threaded view
|

Re: comments on xxxUtf16() I/O calls

Justin Wood (Callek)-2
Aleksey Sanin wrote:

>  > As you pointed out in the bug report,
>> applications and libraries that use NSPR won't be able
>> to benefit from the new functions automatically.
>
> Well, actually I said that some applications/libraries
> will benefit and some will not. E.g. NSS_Init() function
> works just fine with UTF8 strings after the NSPR upgrade
> to handle UTF8 filenames :)
>
> Again, it significantly depends on the application/library.
> If the application/library was written with "current locale"
> in mind then this application will be broken with proposed
> UTF8 upgrade for NSS. But in reality, these applications/libraries
> are already broken to some degree because they are not really
> I18N ready (e.g. no Japanese filename with Chinese default
> locale). Thus, these applications/libraries *have* to be
> changed anyway...
>
> BTW, don't take me wrong. I do understand all the issues
> with ABI/API backward compatibility, etc. I just feel that
> the current NSPR model for handling I18N issues is severely
> broken. And adding another set of calls (Utf8 or Utf16 or ???)
> will make things even worse.
>

Adding another set of calls, and marking the old ones as deprecated
could be a solution, while maintaining the old calls.

The fact is, that NSPR users *can* rely on the native character
encoding.  And saying such NSPR users are inherently broken, is a false
assumption; while many may be, that does not preclude those users who
may have used NSPR on internal applications to a given company/network,
where the developer of the application, is also able to ensure the
environment it runs it meets certain conditions, *such as* is an English
Language system ;-)

It should meet similar constraints as PR_Seek does, (where it is broken
with the potential of large filesizes), but we do not make PR_Seek no
longer work ;-)

~Justin Wood (Callek)

/me hopes he did not mis-state anything.
_______________________________________________
dev-tech-nspr mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-nspr