View Issue Details

IDProjectCategoryView StatusLast Update
0001404DraftPatchpublic2014-05-11 17:45
Reporterulrich1aAssigned Toyorik 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
PlatformallOSOS Version
Product Version 
Target Version0.14Fixed in Version0.14 
Summary0001404: Draft Dimension Override string: Non ascii-Characters are not shown in Drawings
DescriptionAs the summary says: Non ascii-characters are not shown in Drawings. This is a regression after the Pyside migration.
I made a patch, that corrects the behavior.
Steps To Reproduce- Make a dimension.
- Put any Latin1 but non ascii-Character like "Ü" into the override string.
- Put the dimension in a Drawing.
an error message appears about a non asci-character.
TagsNo tags attached.

Relationships

child of 0000853 closedyorik FreeCAD Unicode Strings in Coin3D 

Activities

ulrich1a

2014-02-11 23:05

reporter  

0001-Renables-non-asci-characters-in-the-Override-string-.patch (2,694 bytes)
From 98ad6fe25d30b3c1b1c79ae4169fe6501f05e9e6 Mon Sep 17 00:00:00 2001
From: Ulrich Brammer <ubrammer@t-online.de>
Date: Tue, 11 Feb 2014 23:49:47 +0100
Subject: Renables non-asci characters in the Override string in Drawing
 projections

---
 src/Mod/Draft/Draft.py |   29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/src/Mod/Draft/Draft.py b/src/Mod/Draft/Draft.py
index 55dc894..f083c9a 100644
--- a/src/Mod/Draft/Draft.py
+++ b/src/Mod/Draft/Draft.py
@@ -1636,20 +1636,20 @@ def getSVG(obj,scale=1,linewidth=0.35,fontsize=12,fillstyle="shape color",direct
         return svg
         
     def getText(color,fontsize,fontname,angle,base,text):
-        svg = '<text fill="'
-        svg += getrgb(color) +'" font-size="'
+        svg = u'<text fill="'
+        svg += getrgb(color) +u'" font-size="'
         svg += str(fontsize) + '" '
-        svg += 'style="text-anchor:middle;text-align:center;'
-        svg += 'font-family:'+ fontname +'" '
-        svg += 'transform="rotate('+str(math.degrees(angle))
-        svg += ','+ str(base.x) + ',' + str(base.y) + ') '
-        svg += 'translate(' + str(base.x) + ',' + str(base.y) + ') '
+        svg += u'style="text-anchor:middle;text-align:center;'
+        svg += u'font-family:'+ fontname +'" '
+        svg += u'transform="rotate('+str(math.degrees(angle))
+        svg += u','+ str(base.x) + ',' + str(base.y) + u') '
+        svg += u'translate(' + str(base.x) + u',' + str(base.y) + u') '
         #svg += 'scale('+str(tmod/2000)+',-'+str(tmod/2000)+') '
-        svg += 'scale(1,-1) '
-        svg += '" freecad:skip="1"'
-        svg += '>\n'
-        svg += text
-        svg += '</text>\n'
+        svg += u'scale(1,-1) '
+        svg += u'" freecad:skip="1"'
+        svg += u'>\n'
+        svg += text.decode('utf-8')
+        svg += u'</text>\n'
         return svg
 
         
@@ -3109,8 +3109,9 @@ class _ViewProviderDimension(_ViewProviderDraft):
                 fstring = "%." + str(getParam("dimPrecision",2)) + "f"
             self.string = (fstring % l)
             if obj.ViewObject.Override:
-                self.string = unicode(obj.ViewObject.Override).encode("latin1").replace("$dim",self.string)
-            self.text.string = self.text3d.string = self.string
+                self.string = unicode(obj.ViewObject.Override).encode("utf-8").replace("$dim",self.string)
+            self.text.string = self.string.decode('utf-8').encode('latin1')
+            self.text3d.string = self.string.decode('utf-8').encode('latin1')
 
             # set the distance property
             if round(obj.Distance,precision()) != round(l,precision()):
-- 
1.7.10.4

shoogen

2014-02-12 12:24

developer   ~0004190

have you tested the patch with non latin1 characters? like €

yorik

2014-02-12 12:53

administrator   ~0004193

Thanks! I'll test and apply if all is OK.

ulrich1a

2014-02-12 19:57

reporter   ~0004198

Yes I tested the patch with a non latin1 character. It gives an error at that line where the string is coded to latin1. This is expected and it was the same without the patch. So the non latin1 character is not shown in the 3D-view. (It is a coin-3D-defiency.) But with this patch the character is shown in a Drawing projection. Since the PySide-Migration only ascii-characters are shown in the Drawing projection. So this patch allows utf-8 in a Drawing projection.

I had a discussion with a programmer. He gave me a hint how to avoid the encoding to utf-8. I will add another version of this patch.

yorik

2014-02-12 20:13

administrator   ~0004199

shoogen and I also don't like much turning the SVG output into an unicode object... This might cause problems in other parts of FreeCAD that expect standard strings. We might need to change that.

ulrich1a

2014-02-12 21:15

reporter  

0001-Enable-unicode-in-Drawing-Dimensions.patch (3,034 bytes)
From dc4625b4314cb8a6235c941586048f106be9d30c Mon Sep 17 00:00:00 2001
From: Ulrich Brammer <ubrammer@t-online.de>
Date: Wed, 12 Feb 2014 22:02:54 +0100
Subject: Enable unicode in Drawing Dimensions

---
 src/Mod/Draft/Draft.py |   35 ++++++++++++++++++-----------------
 1 file changed, 18 insertions(+), 17 deletions(-)

diff --git a/src/Mod/Draft/Draft.py b/src/Mod/Draft/Draft.py
index 55dc894..b83355e 100644
--- a/src/Mod/Draft/Draft.py
+++ b/src/Mod/Draft/Draft.py
@@ -1636,20 +1636,20 @@ def getSVG(obj,scale=1,linewidth=0.35,fontsize=12,fillstyle="shape color",direct
         return svg
         
     def getText(color,fontsize,fontname,angle,base,text):
-        svg = '<text fill="'
-        svg += getrgb(color) +'" font-size="'
-        svg += str(fontsize) + '" '
-        svg += 'style="text-anchor:middle;text-align:center;'
-        svg += 'font-family:'+ fontname +'" '
-        svg += 'transform="rotate('+str(math.degrees(angle))
-        svg += ','+ str(base.x) + ',' + str(base.y) + ') '
-        svg += 'translate(' + str(base.x) + ',' + str(base.y) + ') '
+        svg = u'<text fill="'
+        svg += getrgb(color) +u'" font-size="'
+        svg += str(fontsize) + u'" '
+        svg += u'style="text-anchor:middle;text-align:center;'
+        svg += u'font-family:'+ fontname +u'" '
+        svg += u'transform="rotate('+str(math.degrees(angle))
+        svg += u','+ str(base.x) + ',' + str(base.y) + u') '
+        svg += u'translate(' + str(base.x) + u',' + str(base.y) + u') '
         #svg += 'scale('+str(tmod/2000)+',-'+str(tmod/2000)+') '
-        svg += 'scale(1,-1) '
-        svg += '" freecad:skip="1"'
-        svg += '>\n'
+        svg += u'scale(1,-1) '
+        svg += u'" freecad:skip="1"'
+        svg += u'>\n'
         svg += text
-        svg += '</text>\n'
+        svg += u'</text>\n'
         return svg
 
         
@@ -3104,13 +3104,14 @@ class _ViewProviderDimension(_ViewProviderDraft):
             # set text value
             l = self.p3.sub(self.p2).Length
             if hasattr(obj.ViewObject,"Decimals"):
-                fstring = "%." + str(obj.ViewObject.Decimals) + "f"
+                fstring = u"%." + unicode(obj.ViewObject.Decimals) + u"f"
             else:
-                fstring = "%." + str(getParam("dimPrecision",2)) + "f"
-            self.string = (fstring % l)
+                fstring = u"%." + unicode(getParam("dimPrecision",2)) + u"f"
+            self.string = unicode(fstring % l)
             if obj.ViewObject.Override:
-                self.string = unicode(obj.ViewObject.Override).encode("latin1").replace("$dim",self.string)
-            self.text.string = self.text3d.string = self.string
+                self.string = unicode(obj.ViewObject.Override).replace(u"$dim",self.string)
+            self.text.string = self.string.encode("latin1")
+            self.text3d.string = self.string.encode("latin1")
 
             # set the distance property
             if round(obj.Distance,precision()) != round(l,precision()):
-- 
1.7.10.4

ulrich1a

2014-02-12 21:25

reporter   ~0004201

These are different things. You can leave out the "u" before the string objects. Then you have still an asci-string but with an utf-8 encoding in it. This is fully conform to SVG-standards. I did look that up.
There is still the regression, that the non asci-characters are not shown in the override string in a Drawing projection. This is solved by the patch and it should also be solved with a version without the "u"s before the string objects.

So my second version seems of no use. I did not see problems with it so far.

Ulrich

shoogen

2014-02-12 23:43

developer   ~0004203

the problem with the "svg" string being a unicode objects, starts when is has to be processed outside of python. In this case care must be taken when writing to a file. I don't like the fact the the replacement for "$dim" happens on an already decoded string. I haven't spent much time testing. But we should surely offer a (fallback) replacement strategy when encoding to latin1 to avoid encoding errors
t.string = s.decode("utf8").encode("latin1","replace")

ulrich1a

2014-02-15 17:34

reporter  

0002-Latin1-Filter-implementation.patch (1,682 bytes)
From c6c32c25cc6f614151441152e32143ef6b515fb8 Mon Sep 17 00:00:00 2001
From: Ulrich Brammer <ubrammer@t-online.de>
Date: Sat, 15 Feb 2014 00:20:09 +0100
Subject: Latin1 Filter implementation

---
 src/Mod/Draft/Draft.py |   14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/src/Mod/Draft/Draft.py b/src/Mod/Draft/Draft.py
index b83355e..0e2d715 100644
--- a/src/Mod/Draft/Draft.py
+++ b/src/Mod/Draft/Draft.py
@@ -3004,6 +3004,16 @@ class _ViewProviderDimension(_ViewProviderDraft):
         self.onChanged(vobj,"FontName")
         self.onChanged(vobj,"ArrowType")
         self.onChanged(vobj,"LineColor")
+
+    def latin1_filter(self, filter_text):
+        latin_text = ''
+        for i in range(len(filter_text)):
+            if ord(filter_text[i])<256:
+               latin_text += filter_text[i]
+            else:
+               latin_text += ' '
+        return latin_text.encode('latin1')    
+
             
     def updateData(self, obj, prop):
         "called when the base object is changed"
@@ -3110,8 +3120,8 @@ class _ViewProviderDimension(_ViewProviderDraft):
             self.string = unicode(fstring % l)
             if obj.ViewObject.Override:
                 self.string = unicode(obj.ViewObject.Override).replace(u"$dim",self.string)
-            self.text.string = self.string.encode("latin1")
-            self.text3d.string = self.string.encode("latin1")
+            self.text.string = self.latin1_filter(self.string)
+            self.text3d.string = self.latin1_filter(self.string)
 
             # set the distance property
             if round(obj.Distance,precision()) != round(l,precision()):
-- 
1.7.10.4

ulrich1a

2014-02-15 17:38

reporter   ~0004224

I made a filter in order to get only latin1 characters into the 3D-presentation. It works for me. (Using the last two patches.) I tested so far with Arial in the font setting. I can save and load the SVG-drawing.
What else needs to be tested?

yorik

2014-03-29 17:32

administrator  

test utf8 in draft.fcstd (44,635 bytes)

yorik

2014-03-29 17:38

administrator   ~0004515

Apparently, starting from Coin3D 4.0 (libcoin80), utf characters are now fully supported. I just committed a change that fixes that, now all texts (Draft Texts and Draft Dimensions) can have utf8 characters, which display coherently in the Properties editor, the 3D view, and the SVG Drawing sheets.

I still left a fallback for people who have an older version of coin.

The annexed test_utf8_in_draft.fcstd file contains example of such texts, which should render 100% correctly with coin4, and possibly with encoding errors on older versions, but at least not crash. Please test!

I'll still leave this issue open in case there are still problems

ulrich1a

2014-03-30 13:19

reporter   ~0004520

I just tested without Coin3D version 4.0. The fallback solution has a problem in the 3D-View. I made an Draft-dimension and put "$dim Ü" in the override. I get "10.0 Ã" in the 3D-view. But the "Ü" is shown correct in a Drawing projection. A step forward, but a small issue is left.

yorik

2014-03-30 15:47

administrator   ~0004521

Ok so the Ü (which is unicode, therefore works ok in the interface and on the svg sheet), must be converted to latin1 when using coin < 4. This is obviously not optimal (some utf8 chars don't exist in latin1) but I suppose it's the best we can do...

yorik

2014-05-03 16:04

administrator   ~0004611

Just committed a fix (commit 32d7839) that takes the coin verison into account, and converts texts to latin1 if coin version is < 4. Please test!
Thanks

yorik

2014-05-11 17:45

administrator   ~0004650

I believe this is now fixed, reopen if needed!

Issue History

Date Modified Username Field Change
2014-02-11 23:05 ulrich1a New Issue
2014-02-11 23:05 ulrich1a File Added: 0001-Renables-non-asci-characters-in-the-Override-string-.patch
2014-02-12 12:24 shoogen Note Added: 0004190
2014-02-12 12:25 shoogen Relationship added parent of 0000853
2014-02-12 12:52 yorik Assigned To => yorik
2014-02-12 12:52 yorik Status new => assigned
2014-02-12 12:53 yorik Note Added: 0004193
2014-02-12 19:57 ulrich1a Note Added: 0004198
2014-02-12 20:13 yorik Note Added: 0004199
2014-02-12 21:15 ulrich1a File Added: 0001-Enable-unicode-in-Drawing-Dimensions.patch
2014-02-12 21:25 ulrich1a Note Added: 0004201
2014-02-12 23:43 shoogen Note Added: 0004203
2014-02-15 17:34 ulrich1a File Added: 0002-Latin1-Filter-implementation.patch
2014-02-15 17:38 ulrich1a Note Added: 0004224
2014-03-08 20:40 yorik Severity major => minor
2014-03-29 17:32 yorik File Added: test utf8 in draft.fcstd
2014-03-29 17:38 yorik Note Added: 0004515
2014-03-29 17:38 yorik Status assigned => feedback
2014-03-29 17:59 yorik Changeset attached => FreeCAD Master master 749a8fa1
2014-03-29 19:49 yorik Project FreeCAD => Draft
2014-03-30 13:19 ulrich1a Note Added: 0004520
2014-03-30 13:19 ulrich1a Status feedback => assigned
2014-03-30 15:47 yorik Note Added: 0004521
2014-04-19 16:48 yorik Changeset attached => FreeCAD Master master 96967a51
2014-05-03 15:48 yorik Product Version trunk =>
2014-05-03 15:48 yorik Target Version => 0.14
2014-05-03 16:04 yorik Note Added: 0004611
2014-05-03 16:28 yorik Changeset attached => FreeCAD Master master 32d7839d
2014-05-11 17:45 yorik Note Added: 0004650
2014-05-11 17:45 yorik Status assigned => closed
2014-05-11 17:45 yorik Resolution open => fixed
2014-05-11 17:45 yorik Fixed in Version => 0.14
2014-05-11 17:45 yorik Relationship deleted parent of 0000853
2014-05-11 17:45 yorik Relationship added child of 0000853