pivy 0.6.2

[triplus]

I think the translate function here should work with (qt4, qt5) X (py2, py3).

In Python 2 such translate function returns str object (a sequence of bytes) and it can be after converted to unicode by using .decode('utf-8') attribute. And this is the problematic part as in Python 3 str object doesn't have .decode('utf-8') attribute anymore. As Python 3 str represents a unicode string by default. Therefore wherever .decode('utf-8') attribute is used for a string in the code ATM that needs to be made compatible with both Python 2 and Python 3. That is what we talked about on how to achieve that.

But if we go a step back. What would stop working if translate function would return unicode object by default? As using:

Code: Select all

.encode('utf-8").decode('utf-8')

Procedure in Python 2 results in unicode object as returned by:

Code: Select all

try:
    _encoding = QtGui.QApplication.UnicodeUTF8
    def translate(context, text):
        "convenience function for Qt translator"
        return QtGui.QApplication.translate(context, text, None, _encoding)
except AttributeError:
    def translate(context, text):
        "convenience function for Qt translator"
        return QtGui.QApplication.translate(context, text, None)

This was taken from DraftGui translate function. And translations seem to work fine when using Python 2 in Draft workbench for me. Therefore what is the reason distinction what translate function should return (based on if Python 2 or Python 3 is used) was made for Arch translate function?

P.S. I guess we could just continue to return unicode as DraftGui translate function does. And therefore we don't need to use .decode('utf-8') attribute anymore on str object when using Python 2. And by not using .decode('utf-8') attribute on str object we get Python 3 compatible code.

[peterl94]

Exactly. I think it would make sense if DraftTools.translate() returned a unicode object, but...

What would stop working if translate function would return unicode object by default?

I have found some freecad functions that do not expect an unicode object in python2. For example:
FreeCAD.ActiveDocument.addObject("Drawing::FeaturePage",translate("Arch","Page")) fails with "ascii codec can't encode..." if you pass it a unicode object that has non ascii characters. Actually, that is the only one I've notice so far. Others such as Console.PrintMessage and Document.openTransaction that I've seen DraftTools.translate used with seem to work with an unicode object.

[looo]

sry, I was wrong in thinking this uses the qt-translate function directly.

This was taken from DraftGui translate function. And translations seem to work fine when using Python 2 in Draft workbench for me. Therefore what is the reason distinction what translate function should return (based on if Python 2 or Python 3 is used) was made for Arch translate function?

I think python2 always used the encode("utf8") in the DraftTools.translate function. This is actually done, because python2 by default does not encode some non ASCII characters, as @peterl94 has shown. But as I guess in the translation such characters can occur (at least I have encountered one in french-language for the start-page) So that is the reason why the encoding has to be done manually with setting the encoding explicitly.

FreeCAD.ActiveDocument.addObject("Drawing::FeaturePage",translate("Arch","Page"))

The mentioned function uses PyArg_ParseTuple(args, "s|sOO", &sType,&sName,&obj,&view). [1] The first and second argument is a s(tring). Regarding the documentation for python2.7[2] this can be a string or unicode:

s (string or Unicode) [const char *]
Convert a Python string or Unicode object to a C pointer to a character string. You must not provide storage for the string itself; a pointer to an existing string is stored into the character pointer variable whose address you pass. The C string is NUL-terminated. The Python string must not contain embedded NUL bytes; if it does, a TypeError exception is raised. Unicode objects are converted to C strings using the default encoding. If this conversion fails, a UnicodeError is raised.

While there are methods to set utf-8-encoding by default on python2 it is not recommended to use them. So I don't think we can simple skip the translate(...).encode("utf8") and use unicode instead.
Why there is a .decode(...) in the arch-workbench is another question. Actually I don't think this is necessary any more. I don't think removing it cause any problems [3]

[1] https://github.com/FreeCAD/FreeCAD/blob ... p.cpp#L227
[2] https://docs.python.org/2/c-api/arg.html
[3] https://github.com/FreeCAD/FreeCAD/comm ... 64a9873c50

[peterl94]

I see, thanks.

Actually I don't think this is necessary any more. I don't think removing it cause any problems

Unless I'm missing something, this is not always true. Try this:

Code: Select all

from PySide import QtGui
su = u'fran\xe7ais'
s = su.encode('utf-8')
QtGui.QMessageBox.information( None , "" , s )
QtGui.QMessageBox.information( None , "" , s.decode('utf-8') )

Maybe not the cleanest solution, but I'm still for adding are own decode function so that I don't have to check what correctly handles encoded strings and unicode.

[looo]

Unless I'm missing something, this is not always true. Try this:

Thanks for the example.

so the best approach is to use unicode by default. (use the python3-case from DraftTools.translate for python2 too) For functions which uses default python encoding we can encode directly in python or c++.

Maybe not the cleanest solution, but I'm still for adding are own decode function so that I don't have to check what correctly handles encoded strings and unicode.

ps.: changing the translate function will break quite some stuff. (For example I don't think str and unicode can be joined by +operator) Maybe adding a py3-reduntant function is really the best way.

Am I right in guessing this will look something like this?

Code: Select all

def str_decode(string):
    if sys.version_info.major < 3:
        return string.decode("utf-8")
    else:
        return string 

[triplus]

Am I right in guessing this will look something like this?

Code: Select all

def str_decode(string):
    if sys.version_info.major < 3:
        return string.decode("utf-8")
    else:
        return string 

Or:

https://forum.freecadweb.org/viewtopic. ... 60#p178430

As for the rest and based on the tests made. if i interpret them correctly this could be checked in addition:

Code: Select all

translate(...).encode("utf8").decode("utf8")

It could tackle this:

Unicode objects are converted to C strings using the default encoding.

After we could do:

Code: Select all

try: # Python 2
    return translate(...).encode("utf8").decode("utf8")
except AttributeError: # Python 3
    return translate(...)

Do everything directly in the translate function. Current translate function and the proposed one assume UTF-8. I guess there is potential this assumption could be challenged if somehow different encoding would get involved. But i guess good luck with tackling that if that ever happens.

[looo]

Or:

Ah I see, you were a bit faster than me.

Code: Select all

return translate(...).encode("utf8").decode("utf8")

looks quite strange Btw, isn't it the same as using the translated unicode directly?
I think this approach will lead to more work as some of the string operations have to be adopted... So I would vote for adding the special decode function.
and replace all
some_string.decode("utf-8") -> string_decode(some_string)

[triplus]

looks quite strange Btw, isn't it the same as using the translated unicode directly?

Likely yes.

So I would vote for adding the special decode function.

Yes if .decode("utf8") attribute must stay in the code at some places this is the way to go. Or i guess to have optional parameter in translate function. And when parameter is provided (whenever .decode("utf8") attribute needs to be used) therefore always return unicode? And if parameter is not provided return unicode (Py3) or byte string (Py2) from the translate function.

[looo]

Or i guess to have optional parameter in translate function.

Tried that: https://github.com/looooo/FreeCAD/commi ... 6fdf2ae339
But there are more use-cases of "decode("utf8"). For example:

https://github.com/FreeCAD/FreeCAD/blob ... i.py#L1376
https://github.com/FreeCAD/FreeCAD/blob ... i.py#L1395

So maybe a decode function would make more sense.

[triplus]

Both should do the same therefore it is up to you.