[Discussion] Drafting 2d annotations for architecture

A forum dedicated to the Draft, Arch and BIM workbenches development.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
realthunder
Veteran
Posts: 2190
Joined: Tue Jan 03, 2017 10:55 am

Re: [Discussion] Drafting 2d annotations for architecture

Post by realthunder »

carlopav wrote: Tue May 19, 2020 4:30 pm
realthunder wrote: Thu May 14, 2020 12:20 am I think we shall aim at being able to handle 10K+ objects per session.
@realthunder, @yorik, is this thing onekk is describing something that make sense to you?
Like I said before, I think it is better to handle lines and stuff inside a single a single Draft object, so you can use more efficient object model. FreeCAD objects may slow down the system because of all those onChanged() signaling. That being said, a database is probably an overkill for this.

BTW, I have actually made a few optimization on tree view, document loading and stuff for handling large amount of objects. I've tested with projects containing dozens of files, over 2k objects, loading in about 5 seconds. Most of the time is actually spent on loading the workbench for the first time. If those workbenches are preloaded, then loading time is less than 3 seconds. The loading time also depends on the content of the object, for example if those objects has complex geometries, etc.
Try Assembly3 with my custom build of FreeCAD at here.
And if you'd like to show your support, you can donate through patreon, liberapay, or paypal
User avatar
onekk
Veteran
Posts: 6222
Joined: Sat Jan 17, 2015 7:48 am
Contact:

Re: [Discussion] Drafting 2d annotations for architecture

Post by onekk »

So the idea of having a mean to add a sqlite file in the FCstd format is not too strange, and if ir used by a GIS program, that usually manage thousand of objects, will be a good solution, not said that the relevant libraries are already included in python, so non need to search anything strange.

What do you think?

Regards

Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.

Blog: https://okkmkblog.wordpress.com/
User avatar
yorik
Founder
Posts: 13665
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: [Discussion] Drafting 2d annotations for architecture

Post by yorik »

@carlopav : Didn't meant to disrespect 2D ;) I also think the act of "drafting/drawing/annotating" is needed everywhere, and also look at your model in plan or section very fundamental when doing architecture. I just don't like the "here we work in plan and sections! Ah, you want a 3D view? Really?" paradigm of Revit. A great thing of FreeCAD is that it's totally manageable to do almost anything directly in the 3D view that would be almost impossible in Revit (you wouldn't be able to see what you are doing). We should take great care of that.

@onekk: In principle anything can be embedded in a FreeCAD file. That's why the file format is designed that way. But I also think a database is overkill to solve an internal problem, unless you specifically need a database, like GIS. I would find it totally sound to have a GIS workbench save a database file inside a FreeCAD file. But then that GIS WB is sole responsible for dependencies, libraries, and all it needs.

Let's start designing such a generic, multi-annotation (Draft) annotation! I think it is a very good path to go.

I think it should:

1) not be shape-based, as there is no need for shapes here (no booleans, etc)
2) contain "subobjects" like texts, dimensions, and generic geometry (that can yes be obtained from shapes)
3) have a very nice and well-designed task UI and toolbar. Maybe actually it should be an own workbench? Like sketcher? That is activated on edit?
4) coded in C++? I would find that pretty interesting for the speed, and if we start a separate WB, we can start from scratch really nicely. I did something very similar when we started coding Path, started with designed the "invisible" subobjects (Path command, path tool, etc...) then the FreeCAD docobject that holds them all, then the view provider, then the UI... I think it proved pretty solid
User avatar
onekk
Veteran
Posts: 6222
Joined: Sat Jan 17, 2015 7:48 am
Contact:

Re: [Discussion] Drafting 2d annotations for architecture

Post by onekk »

yorik wrote: Wed May 20, 2020 9:00 am @onekk: In principle anything can be embedded in a FreeCAD file. That's why the file format is designed that way. But I also think a database is overkill to solve an internal problem, unless you specifically need a database, like GIS. I would find it totally sound to have a GIS workbench save a database file inside a FreeCAD file. But then that GIS WB is sole responsible for dependencies, libraries, and all it needs.
Yes, but someone has told that there will be problems fi there are many objects, why don't use the database interface, a carefully created database could hold thousands of annotations and permit complex queries, and maybe some other nice things, like updating reference, and maybe searching annotations with conditionals.

a simple query could be done even in python, but for a complex query in say 10.000 annotations, maybe even the C++ code is complex to develop: let me explain.

Code: Select all

import sqlite3 as sq3
import json

conn = sq3.connect('/home/carlo-arch/annotations.db')
c = conn.cursor()


def create_db():
    """Create table"""
    c.execute("CREATE TABLE annotations (Id INTEGER PRIMARY KEY, name, content, position, style)")
    conn.commit()

def populate():
    cnt = 0
    
    for idx in range (0,1000):
        cnt += 1
        name = "ann_" + str(idx)
        content = "contenuto dell'annotazione {0}".format(name)
        pos = json.dumps((idx, idx, idx))
        style = "style_{0}".format(str(cnt)) 
        c.execute("insert into annotations (name, content, position, style) values (?,?,?,?)", (name, content, pos, style) ) 
        if cnt > 3:
            cnt = 0    
    conn.commit()

def retrieve():
    for row in conn.execute("SELECT * FROM annotations WHERE style LIKE 'style_1'"):
        print(row[0], row[1], row[2], json.loads(row[3]), row[4])

#c.execute("DROP TABLE annotations")
#create_db()
#populate()

retrieve()

for testing you have to run create.db and modify the path of the database, in conn = sq3.connect('/home/carlo-arch/annotations.db')

once you have created the table, feel free to modify the populate() and then launch it, or decomment both create and populate, for the first run.

Now you have the database, of 1000 or 10000 or even 100000 annotation, the table is simple, a name, holding:

- Id as index that could be generated in a automated way due to the "PRIMARY KEY" value in the creation table
- name that could be a meaningful name and could be used as a descriptor
- content the content of the annotation, watever you want
- pos a field that hold a tuple that is serialized using json, so it could hold whatever you want, maybe directly a Vector, we could try
- style a reference to the style_name, the proper style definition could be put maybe in another table where style_name could be the primary key

now after having created those hundreds of annotations, yoiu could retrieve them using simply a query like:

.execute("SELECT * FROM annotations WHERE style LIKE 'style_1'"):

if you want search in a big project some other things, maybe the author in a collaborative environment, add a author field and you could sort using conditionals using the style and the author.

writing simply a string "SELECT * FROM annotation WHERE style LIKE 'style_1" AND author LIKE 'yorik'", and then simply iterate in the row objects of the returned query.

No need to write if then else or so.

yorik wrote: Wed May 20, 2020 9:00 am Let's start designing such a generic, multi-annotation (Draft) annotation! I think it is a very good path to go.

I think it should:

1) not be shape-based, as there is no need for shapes here (no booleans, etc)
2) contain "subobjects" like texts, dimensions, and generic geometry (that can yes be obtained from shapes)
3) have a very nice and well-designed task UI and toolbar. Maybe actually it should be an own workbench? Like sketcher? That is activated on edit?
4) coded in C++? I would find that pretty interesting for the speed, and if we start a separate WB, we can start from scratch really nicely. I did something very similar when we started coding Path, started with designed the "invisible" subobjects (Path command, path tool, etc...) then the FreeCAD docobject that holds them all, then the view provider, then the UI... I think it proved pretty solid
storing the important information of an annotation could even create them on the fly in the final document, if needed.

Not telling that a BLOB field in sqlkite coudl hold a 1MB binary data that maybe could contain directly the object if it serialized in some way.

Regards

Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.

Blog: https://okkmkblog.wordpress.com/
User avatar
regis
Posts: 747
Joined: Sun Jul 12, 2015 8:17 am
Contact:

Re: [Discussion] Drafting 2d annotations for architecture

Post by regis »

paullee wrote: Tue May 19, 2020 6:24 pm
regis wrote: Tue May 19, 2020 12:21 am
paullee wrote: Mon May 18, 2020 3:43 pm On my Fedora 31, it is just ' ' when nothing is input - so I have no idea which Font it is using
I meant changing the font of any darft annotation text or draft dimensions. is that working on your side?
Have a quick test - it seems it works :)


Screenshot from 2020-05-20 02-21-04.png
Screenshot from 2020-05-20 02-21-13.png
Screenshot from 2020-05-20 02-21-22.png
Screenshot from 2020-05-20 02-21-40.png
Screenshot from 2020-05-20 02-21-51.png
Indeed i see it works for you, but not for me.
carlopav
Veteran
Posts: 2062
Joined: Mon Dec 31, 2018 1:49 pm
Location: Venice, Italy

Re: [Discussion] Drafting 2d annotations for architecture

Post by carlopav »

@onekk I have no experience to understand if there is some hidden potential that Yorik and realthunder do not spot in the approach you are suggesting abut using a database for annotations... Let's think about that some more time...
yorik wrote: Wed May 20, 2020 9:00 am @carlopav : Didn't meant to disrespect 2D ;)
I know :) I just wanted to state it is fundamental to me.
And I also agree that everything has to happen inside the 3d view, since that's the true meaning of a plan, to be a schematic representation of a plane cutting the 3d space! What is more satisfying than seeing this happen?
4) coded in C++? I would find that pretty interesting for the speed, and if we start a separate WB, we can start from scratch really nicely. I did something very similar when we started coding Path, started with designed the "invisible" subobjects (Path command, path tool, etc...) then the FreeCAD docobject that holds them all, then the view provider, then the UI... I think it proved pretty solid
This perspective seems really promising. What about completely discarding the z coordinate for those subobjects?
follow my experiments on BIM modelling for architecture design
User avatar
onekk
Veteran
Posts: 6222
Joined: Sat Jan 17, 2015 7:48 am
Contact:

Re: [Discussion] Drafting 2d annotations for architecture

Post by onekk »

After having thought on the db approach, I've envised some critical points, that could be put in the discussion.

Developer are facing generally the problem of speed when doing searches, for which a relational db like sqlite is tailored for this scope, so qhy not leverage his power in doing such things.

Said so the main concern of a developer could be "the binary format could lead to data corruption", the answer is yes.

But if we are dealing with thousand of objects maybe a search would be using much horsepower, using the object directly.

Why not leveraging the power of sqlite to construct some "pivot table" between relevant data and the real objects?

think a document with thousand of objects, and maybe you will search for a particular object that have one property that interest you.

maybe simply you want to find all objects, with a particular name or whose position is between a min point and a max point.

if you build a table containing the object data and the object relevant reference, like this:

CREATE TABLE objects (Id INTEGER PRIMARY KEY, name, fcid, label, bbmin, bbmax, ..... )

fcid is the internal FreeCAD id number, or maybe the unique name property, i.e the identifier for this objects that permit to retrieve it correctly by the subsequent code.

then you make a slection like:

SELECT * FROM objects WHERE name LIKE "whatyouwant" AND bbmin > yourminpos AND bbmax < yourmaxpos"

you will find all the relevant object and maybe load in memory only these objects.

This pivot table could be stored in the file, or maybe build at the loading of file.

Same thing should be done with annotations, maybe an XML file, or a csc file could hold the data (for an easy "crash recovery") and a pivot table stored in the file, or rebuild on loading.

eventually a mean to rebuild such tables could be supplied.

When dealing with thousand of objects, this will be a correct approach, a similar approach is used in BRL-CAD (I know it is jurassic but it have been built for dealing with complex objects) it use a database for holding all the 3d objects and use this when for example it id doing ballistic analysis (it was build by the US army in fact his name mean Ballistic Research Laboratory - CAD).

Anyone who has programmed a complex search could catch the amount of work needed to put in place a query, with sqlite is only matter of doing a query and iterate through the resulting list of database rows, not telling that the selct could only return the relevant internal id to retrieve the objects, using:

SELECT fcid from objects ....

Regards

Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.

Blog: https://okkmkblog.wordpress.com/
User avatar
Kunda1
Veteran
Posts: 13434
Joined: Thu Jan 05, 2017 9:03 pm

Re: [Discussion] Drafting 2d annotations for architecture

Post by Kunda1 »

@onekk I haven't read this thread in details but JFYI have you seen: https://github.com/furti/FreeCAD-Reporting "SQL Like Reporting for FreeCAD Documents" ?
Alone you go faster. Together we go farther
Please mark thread [Solved]
Want to contribute back to FC? Checkout:
'good first issues' | Open TODOs and FIXMEs | How to Help FreeCAD | How to report Bugs
User avatar
onekk
Veteran
Posts: 6222
Joined: Sat Jan 17, 2015 7:48 am
Contact:

Re: [Discussion] Drafting 2d annotations for architecture

Post by onekk »

Kunda1 wrote: Thu May 21, 2020 10:12 am @onekk I haven't read this thread in details but JFYI have you seen: https://github.com/furti/FreeCAD-Reporting "SQL Like Reporting for FreeCAD Documents" ?
No I haven't read it, but the approach is different, here we are discussing about to imlement a sqlite database to manage the possibly thousand annotations in a big project, the example project from which at least I'm thinking is a multi plane building, with one or maybe multiple 2d "floorplans" with a bunch of lines and annotations (think maybe as HVAC or wiring harnesses) with many annotations on them.

So maybe with 10 or 12 plans, and 3 2d drafting you have thousand of annotations.

We are speaking of leveraging the relational database module based on sqlite3 "on board" of python 3 to manage the searching or maybe to hold the entire annotations structure.

Maybe during the saving of the work in FCstd file some of this structure could be translated in some other format, XML, or even csv.

During the opening phase of the FCstd file, the data are put in the sqlite3 structure to permit fast queries and updating the structure using the sql syntax.

Some ideas are arising, so we have to decide one way to spedd up such project, someone has proposed to sepdd it up using C++ interfaces, but for not reinventing the wheel, I've proposed to leverage the "ready made" python sqlite3 interface already "on board" as a standard library in python.

Hope to have been almost clear.

Carlo D.
GitHub page: https://github.com/onekk/freecad-doc.
- In deep articles on FreeCAD.
- Learning how to model with scripting.
- Various other stuffs.

Blog: https://okkmkblog.wordpress.com/
User avatar
yorik
Founder
Posts: 13665
Joined: Tue Feb 17, 2009 9:16 pm
Location: Brussels
Contact:

Re: [Discussion] Drafting 2d annotations for architecture

Post by yorik »

About databases: You might have a point there, working with some kind of database structure *could* mean speed improvements over the current way FreeCAD works, but if we would want to apply such thing to whole FreeCAD it would mean a complete rewrite of very large and deep parts of FreeCAD itself. I don't see that even remotely likely to happen. And doing it just for one small part (annotations) seems to me a huge work when using existing tools is pretty straightforward, specially that there is no guarantee it would effectively bring any useful improvement.

If you want to pursue this path and build something to test, it would definitely be interesting. But I guess it would be your job to prove your point there ;)
Post Reply