mergeProject: Problem if path has forward and backslashes

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Post Reply
User avatar
Roy_043
Veteran
Posts: 8450
Joined: Thu Dec 27, 2018 12:28 pm

mergeProject: Problem if path has forward and backslashes

Post by Roy_043 »

Code: Select all

path = "D:/FreeCAD/PR-arch-window\\testwindow.FCStd"
FreeCADGui.doCommand("FreeCADGui.ActiveDocument.mergeProject('" + path + "')")
Error:

Code: Select all

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<string>", line 1, in <module>
Base.FreeCADError: ios_base::failbit set: iostream stream error
Version:

Code: Select all

OS: Windows 8.1 Version 6.3 (Build 9600)
Word size of FreeCAD: 64-bit
Version: 0.20.26858 (Git)
Build type: Release
Branch: master
Hash: e209bc706d35121098f9bac779bc6b09c24ddd95
Python version: 3.8.6+
Qt version: 5.15.2
Coin version: 4.0.1
OCC version: 7.5.3
Locale: Dutch/Netherlands (nl_NL)
kisolre
Veteran
Posts: 4164
Joined: Wed Nov 21, 2018 1:13 pm

Re: mergeProject: Problem if path has forward and backslashes

Post by kisolre »

Here are my results trying to merge an existing file:

Code: Select all

>>> FreeCADGui.ActiveDocument.mergeProject('E:\_Projects\_TempTests\2.FCStd')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
Base.FreeCADError: ios_base::failbit set: iostream stream error
>>> FreeCADGui.ActiveDocument.mergeProject('E:/_Projects/_TempTests/2.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('E:\_Projects\_TempTests/2.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('E:\_Projects/_TempTests/2.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('E:\_Projects\_TempTests\2.FCStd')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
Base.FreeCADError: ios_base::failbit set: iostream stream error
Looks like only all forward slashes give error and mixtures dont.
OS: Windows 8.1 Version 6.3 (Build 9600)
Word size of FreeCAD: 64-bit
Version: 0.20.27078 (Git)
Build type: Release
Branch: master
Hash: b836712c83556b4f30d3404e8ab9f5192979561c
Python version: 3.8.8
Qt version: 5.15.2
Coin version: 4.0.1
OCC version: 7.5.3
Locale: Bulgarian/Bulgaria (bg_BG)
User avatar
Roy_043
Veteran
Posts: 8450
Joined: Thu Dec 27, 2018 12:28 pm

Re: mergeProject: Problem if path has forward and backslashes

Post by Roy_043 »

Code: Select all

# Your tests:
FreeCADGui.ActiveDocument.mergeProject("D:/FreeCAD/PR-arch-window/testwindow.FCStd")  # OK.
FreeCADGui.ActiveDocument.mergeProject("D:\FreeCAD\PR-arch-window/testwindow.FCStd")  # OK.
FreeCADGui.ActiveDocument.mergeProject("D:\FreeCAD/PR-arch-window/testwindow.FCStd")  # OK.
FreeCADGui.ActiveDocument.mergeProject("D:\FreeCAD\PR-arch-window\testwindow.FCStd")  # Error.

# New tests:
FreeCADGui.ActiveDocument.mergeProject("D:/FreeCAD/PR-arch-window\testwindow.FCStd")  # Error.
FreeCADGui.ActiveDocument.mergeProject("D:/FreeCAD/PR-arch-window\\testwindow.FCStd") # OK.
I confirm your results, but not your conclusion (note that what you call a "forward slash" is in fact a "backslash").
An (unescaped) backslash before the filename is causing the problem I think.

To make the code in my OP work you have to use 4 blackslashes:

Code: Select all

path = "D:/FreeCAD/PR-arch-window\\\\testwindow.FCStd"
FreeCADGui.doCommand("FreeCADGui.ActiveDocument.mergeProject('" + path + "')")
aapo
Posts: 615
Joined: Mon Oct 29, 2018 6:41 pm

Re: mergeProject: Problem if path has forward and backslashes

Post by aapo »

You have multiple consecutive backslashes in your path, which is an unsupported and undocumented Windows operating system feature, and it may or may not work depending on the application and the OS. On the other hand, multiple forward slashes are a POSIX-standard, and they must be supported and treated equally to one forward slash, except in the beginning of the path two forward slashes can (but not must) have a special meaning.

It seems that multiple consecutive backslashes in a path is an undefined behavior under the standards; and it's not a bug if they don't work (but they may work with some programs, of course, whereas other show error messages). Multiple consecutive forward slashes in a path is standardized, and should work according to the POSIX standard: "Multiple successive slashes are considered to be the same as one slash."

https://stackoverflow.com/questions/330 ... dows-paths
https://unix.stackexchange.com/question ... e-username
chrisb
Veteran
Posts: 53933
Joined: Tue Mar 17, 2015 9:14 am

Re: mergeProject: Problem if path has forward and backslashes

Post by chrisb »

aapo wrote: Tue Jan 18, 2022 8:35 pm It seems that multiple consecutive backslashes in a path is an undefined behavior under the standards; and it's not a bug if they don't work
I guess that the OS doesn't see the double backslashes, they rather work like quoted single backslashes.
A Sketcher Lecture with in-depth information is available in English, auf Deutsch, en français, en español.
User avatar
Roy_043
Veteran
Posts: 8450
Joined: Thu Dec 27, 2018 12:28 pm

Re: mergeProject: Problem if path has forward and backslashes

Post by Roy_043 »

aapo wrote: Tue Jan 18, 2022 8:35 pm You have multiple consecutive backslashes in your path
Using double backslashes in strings is required in certain programming languages.

In the example with the 4 backslashes only one backslash is sent to the OS. But please forget about that example as it is confusing the issue.

Code: Select all

>>> FreeCADGui.doCommand("FreeCADGui.ActiveDocument.mergeProject('D:/FreeCAD/PR-arch-window\\\\testwindow.FCStd')")
>>> FreeCADGui.ActiveDocument.mergeProject('D:/FreeCAD/PR-arch-window\\testwindow.FCStd')
aapo
Posts: 615
Joined: Mon Oct 29, 2018 6:41 pm

Re: mergeProject: Problem if path has forward and backslashes

Post by aapo »

Roy_043 wrote: Wed Jan 19, 2022 9:32 am In the example with the 4 backslashes only one backslash is sent to the OS. But please forget about that example as it is confusing the issue.
You're correct, that's not the source of the problem. That said, I cannot reproduce the problem with Linux, everything works as expected, and I can reproduce the fail with single backslash (escaping the next char) with Linux, too. I tested with two files, "/tmp/test.FCStd" and "/tmp/test/test.FCStd" and all combinations of even number of backslashes seem to work. (Note the single backslash in the failing attempt below)

Code: Select all

>>> FreeCADGui.ActiveDocument.mergeProject('/tmp\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('/tmp\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('\\tmp\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('\\tmp/test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('\\tmp\\\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('//tmp\\\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('/tmp/test\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('/tmp/test\\\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('/tmp\test\\\\test.FCStd')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
Base.FreeCADError: basic_ios::clear: iostream error
>>> FreeCADGui.ActiveDocument.mergeProject('/tmp\\test\\\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('/tmp/test\\\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('\\tmp/test\\\\test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('/tmp\\test/test.FCStd')
>>> FreeCADGui.ActiveDocument.mergeProject('/////tmp/////////test\\test.FCStd')
>>> 

OS: Debian GNU/Linux bookworm/sid (KDE/plasma)
Word size of FreeCAD: 64-bit
Version: 0.20.27100 (Git)
Build type: Release
Branch:
Hash: b9280ae760182783f74eddc77a114686c4143e00
Python version: 3.9.10
Qt version: 5.15.2
Coin version: 4.0.0
OCC version: 7.5.1
Locale: English/United States (en_US)
Last edited by aapo on Wed Jan 19, 2022 7:26 pm, edited 1 time in total.
aapo
Posts: 615
Joined: Mon Oct 29, 2018 6:41 pm

Re: mergeProject: Problem if path has forward and backslashes

Post by aapo »

There is one suspicious thing in mergeProject() in DocumentPyImp.cpp, namely Python argument parsing in the first if-clause.

Code: Select all

PyObject*  DocumentPy::mergeProject(PyObject * args)
{
    char* filename;
    if (!PyArg_ParseTuple(args, "s", &filename))     // convert args: Python->C
        return NULL;                             // NULL triggers exception

    PY_TRY {
        Base::FileInfo fi(filename);
        Base::ifstream str(fi, std::ios::in | std::ios::binary);
        App::Document* doc = getDocumentPtr();
        MergeDocuments md(doc);
        md.importObjects(str);
        Py_Return;
    } PY_CATCH;
}
The Python documentation in https://docs.python.org/3/c-api/arg.html says about the 's' conversion that
Note

This format does not accept bytes-like objects. If you want to accept filesystem paths and convert them to C character strings, it is preferable to use the O& format with PyUnicode_FSConverter() as converter.
This implies that filesystem paths should be converted with 'O&' style, and I modified and compiled the source with this change. Unfortunately, on Linux, it seems both ways work ok, so I'm not able to debug the problem as I can't make it fail. But if anyone being able to compile with windows could try to replace

Code: Select all

if (!PyArg_ParseTuple(args, "s", &filename))     // convert args: Python->C
on line 178 of DocumentPyImp.cpp with

Code: Select all

if (!PyArg_ParseTuple(args, "O&", &PyUnicode_FSConverter, (void *)&filename))
and recompile, and see if it fixes the problem. The 'O&' version should be the correct one according to the documentation, and there are other places in the same class that should be fixed, too. But let's first see if this is the problem, or if it's still something else.
Post Reply