bugFree UCS Outline Fonts - Bugs: bug #42806, Missing Devanagari ligature...

 
 

bug #42806: Missing Devanagari ligature r+vocalic r for Sanskrit

Submitter:  François Patte <frpatte>
Submitted:  Fri 18 Jul 2014 03:49:05 PM UTC
   
 
Category:  individual character(s) Severity:  3 - Normal
Item Group:  glyph(s) incorrect Status:  Fix posted
Privacy:  Public Assigned to:  Stevan_White
Open/Closed:  Open Release:  * development
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 27 Aug 2014 07:44:14 AM UTC, comment #29: 

Attached is the passage in Renou's Grammaire Sanscrite, rendered in Firefox with FreeSerif.


Steve White <Stevan_White>
Group administrator
Sun 24 Aug 2014 12:21:25 PM UTC, comment #28: 

Posted further improvements regarding vocalics and vocalic marks.

Besides the four ra + vocalic-mark combinations (as apparently used by the velthuis system), combinations of ra + virama + vocalic also work in most fonts (and I am told they will be standardized in upcoming Unicode).

Furthermore, there is the distinction between ka with l-vocalic-mark beneath and kla with r-vocalic beneath; they should be distinct (although the former appears in Sanskrit only in words with 'klrp', and I'm not sure if the latter appears at all.)

see attachments.

(file #31946, file #31947, file #31948, file #31949)

Steve White <Stevan_White>
Group administrator
Sun 17 Aug 2014 02:31:17 PM UTC, comment #27: 

Posted a modification of François's original tex file, showing current state of FreeFont, as well as several other Devanagari fonts.  Note that some fonts require parameters to turn on the special features (especially, but not exclusively, FreeFont).

I've found nowhere that all these forms are documented together. Only the ra-rvocalicsign is in general use, but the la-lvocalicsign is also seen (although it seems to be rendered the same as U+090C).

I am now convinced beyond reasonable that there is a long tradtion of regarding these forms as semivowel + vocalic-sign, mostly by the order that words containing them appear in 19th century dictionaries. 

I am also inclined to regard the 3-character forms with virama as an accident of OpenType engines.  I haven't seen a Sanskrit search engine yet that supports these encodings.  Unicode does not say that ra + virama + letter should put a reph on the letter unless the letter is a consonant.

(file #31910, file #31911)

Steve White <Stevan_White>
Group administrator
Thu 14 Aug 2014 07:48:04 PM UTC, comment #26: 

Shree has pointed out that in
the Unicode Book 6.2, Ch 9
    http://www.unicode.org/versions/Unicode6.2.0/ch09.pdf
    Page 290 Table 9.3
the second entry from the last is ra + r-vocalic-sign, showing a ligature resembling r-vocalic with a reph.

This would seem to legitimize the construct within Unicode.


Steve White <Stevan_White>
Group administrator
Wed 13 Aug 2014 11:45:53 PM UTC, comment #25: 

Hi, Shree,

Several things going on here, there are several questions.

Everything I have tried, including FreeFont for some time, supports the ra + virama + vocalic to put a reph above a vocalic letter.  So long as the vocalics are regarded as consonants, this is according to Unicode rules.  Really there is no question about this.

But that is not the question being asked here.

The question is (as you put it) how to display ra + vocalic-sign combinations?

It is like: how to display a consonant conjunct?  In Hindi, some might be written as a half form followed  by the consonant, others by a ligature.  In Sanskrit, as we have discusssed before, the set of conjucts normally written as ligatures is different from that of modern Hindi.  And Unicode does not attempt to standardize this.

This is a job for one of those script grammars.  And this is something I have yet to see.

I really need an answer to the question: can one tell from the order in which words appear in a dictionary, that the voclic with reph above is to be understood as ra + virama + vocalic, or as ra + vocalic-sign?  I think that would settle it.

If they are ordered in the dictionary as though written ra + vocalic-sign, then that encoding should produce the desired combination.

Are you saying that 'old' dictionaries such as MW order the words one way, and 'modern' dictionaries order them another way?
Are you thinking dictionaries should be rearranged to comply with Unicode?  I'm afraid I would strongly disagree with that.



Steve White <Stevan_White>
Group administrator
Wed 13 Aug 2014 03:30:41 PM UTC, comment #24: 

Steve,

If I remember correctly, I think you had mentioned in the beginning when the discussion started that this construct was there in one of the styles - calcutta, I think.

It would be wise to offer both choices as different styles for  sanskrit, one as shown in the 'old' MW dictionary and the other 'modern' unicode option.

Thanks!

Shree Devi Kumar <shree>
Wed 13 Aug 2014 06:33:40 AM UTC, comment #23: 

Another commit to SVN gives perhaps better behavior with vowel signs.

I'm not sure how vowel signs should interact with this construct though, or even if it ever occurs with vowels in real words.

Further, after substantial experimenting, I couldn't make it work like other consonant+reph conjuncts; I suspect that the font shaping engine doesn't treat the vocalic characters as other consonants.

Steve White <Stevan_White>
Group administrator
Mon 11 Aug 2014 07:23:27 PM UTC, comment #22: 

Hi François,

Sorry, I should have mentioned that the language text must be specified to be Sanskrit.  See attachments.

I have been discussing this offline with some very good people including Shree, and some of them argue that this construct is just wrong -- that only the Unicode standard encoding (with virama) should be used.  I am very unclear on where this other encoding comes from, only that it has existed for a few years, at least in Velthuis.

So I'm not sure the feature will stay.   You can use it for now, but it's just for testing.

(file #31886, file #31887)

Steve White <Stevan_White>
Group administrator
Mon 11 Aug 2014 04:20:06 PM UTC, comment #21: 

Hi Steve,

I updated my svn directory to revision 2952 this morning and did not see any change for ra+vocalic_r ligature...

François Patte <frpatte>
Sun 10 Aug 2014 09:01:11 PM UTC, comment #20: 

Posted fix to Serif and Sans with their bold faces.

Added four repha forms encoded with ra + vocalic sign, one for each of the signs r-vocalic, rr-vocalic, l-vocalic, and ll-vocalic, for Sanskrit only.

Shree indicated a plausible traditional need for them: that the dictionary order for some words beginning with these signs would indicate spelling with this encoding.  (I would still like to see rather more definite evidence or documentation, but for now, it's good enough for me to take action.)

These forms are also found in the Sanskrit fonts Siddhanta and Sanskrit 2003, although not in other popular Hindi fonts.  As noted, they are also supported by the Velthuis system.

Cheers!

Steve White <Stevan_White>
Group administrator
Thu 07 Aug 2014 03:12:26 AM UTC, comment #19: 

Hi Shree,

The passage, as I understand it, lists a few examples, and does not claim to be exhaustive.

Furthermore, it is talking about not only combinations with r_vocalic, but with vowels -- it would seem generally, some consonant combinations with some below-base marks take on special forms.

So far, we have

  • this is a Sanskrit-only formation
  • it involves ra + r_vocalic and rr_vocalic, perhaps l_vocalic


Instead of seeing this as an alternate encoding for a given conjunct character, this should be seen as special forms of combination of consonants with below-base forms.

The question remains: exactly which combinations are involved?
(The passage you provided states that it is not complete.)

I should say, I have now seen some of these others in Sanskrit fonts (e.g. ha + r_vocalic).

It would be good for a listing of these forms to appear in some modern "script grammar" for Sanskrit.

Steve White <Stevan_White>
Group administrator
Sun 03 Aug 2014 10:00:27 AM UTC, comment #18: 

I asked my son to translate https://savannah.gnu.org/bugs/download.php?file_id=31750 for me.

"In some of these combinations, the vowel's sign (or that of the consonant) is a little changed from how it normally appears - Example : ru, ruu, hu, huu, hR rR, (this is contrary to the usual method of using them, and is analogous to how r is written when before a consonant. d and c before u, uu, R take the same alterations seen here."

So, it seems to me that the passage in French is showing the exceptions - the ligatures for the combinations where the ligature form is different from the usual rule.

Shree Devi Kumar <shree>
Sun 03 Aug 2014 09:56:20 AM UTC, comment #17: 

Hi Steve,
Also posting here in addition to email reply regarding this ..

Here are links to scans from sanskrit dictionary pages .

Please see:
http://www.sanskrit-lexicon.uni-koeln.de/cgi-bin/serveimg.pl?file=/scans/MDScan/MDScanjpg/mcd148--naipuNya.jpg
and
http://www.sanskrit-lexicon.uni-koeln.de/cgi-bin/serveimg.pl?file=/scans/MDScan/MDScanjpg/mcd142--nirabhiprAya.jpg
for entries for
nair-rita
and
nir-riti

The above two pages are http://www.sanskrit-lexicon.uni-koeln.de/scans/MDScan/index.php?sfx=jpg - dictionary from 1893.

Also see nir-ri in
http://www.sanskrit-lexicon.uni-koeln.de/scans/MWScan/index.php?sfx=jpg
monier williams dictionary - first edition 1899 - scanned copy is from 1964.

The dictionary order of where these combinations occur shows that it is 'r' consonant followed by 'ri' vowel. ie  ra+rvocalic.


Shree Devi Kumar <shree>
Sun 27 Jul 2014 03:39:53 PM UTC, comment #16: 

Hi François,

Your comment about whether the compound is preceeded by a space or vowel sounds like the kind of justification that would suffice, although I do not yet understand it.

(If you could produce some real words, that would be great -- you know there are some Sanskrit texts out there... )

I'm still looking for a standard -- the TDIL has produced very helpful standards for many modern Indic languages, but I haven't yet found one for Sanskrit/Devanagari.

Steve White <Stevan_White>
Group administrator
Sat 26 Jul 2014 04:29:26 PM UTC, comment #15: 

Hi Steve,

I don't know much about font mechanisms to create ligatures but I could make some tests regarding this r.r ligature (I used Emacs with ucs-insert).

0x0930+0x094d+0x090b produces the right ligature using Freeserif fonts.

Now the problem is: if this ligature is preceeded by a space (beginning of a word) or a vowel (āvirṛjīka, nirṛta ...) the ligature 0x0930+0x094d+0x090b must be used. If the ligature is preceeded by a consonnant, the ligature between the consonnant and r must be done first, then 0x0943 must be added (unfortunately I have no example right now...)

I could add some rules in the velthuis-sanskrit.map file to produced this behaviour with the FreeSerif fonts (and other fonts as well) but I am not sure that these rules have no side effects...

I had a look in Sankrit2003 fonts, the sought ligature is given in the "private area" of the font.

François Patte <frpatte>
Mon 21 Jul 2014 06:01:22 PM UTC, comment #14: 

The "AND" remark is bunged up.  Try again:

AND I've found that also the combination 0x0930+0x0944 ("r.r" ?) also makes a repha with the Sanskrit fonts, while 0x0932+0x0943; ("r.l" ?) 0x0932+0x0944; ("r.L" ?) both fail to produce repha.

(I'm just guessing about the velthuis transcriptions -- I can't seem to get velthuis working on your example or on my own... Funny, no error messages, just the text isn't transcribed into Devanagari.)

Steve White <Stevan_White>
Group administrator
Mon 21 Jul 2014 05:22:37 PM UTC, comment #13: 

Hi François,

I appreciate the advantages of Velthuis, and I'm familiar with it (FreeSerif is based on the glyphs from the velthuis MetaFont).

Yes my r.k was a typo.  I think I meant "rk".

But this is the point:  velthuis "rk" seems to produce the 3-character encoding 0x0930+0x094D+0x0915, for which all reasonable fonts have instructions to make a ka with a repha.  However velthuis "r.R" seems to instead produce a 2-character encoding 0x0930+0x090B.  This is unlike the description in chapter 9 of the Unicode book for producing a repha.

However: so far, every font I've seen meant for Hindi does not support this 2-character thing, while every font (except FreeFont) meant for Sanskrit does.

AND I've found that also the combination 0x0930+0x0960 ("r.r" ?) also makes a repha with the Sanskrit fonts, while 0x0930+0x090C ("r.l" ?) 0x0930+0x0961 ("r.L" ?) both fail to produce repha.

So... what gives?  Where is this behavior documented?

Steve White <Stevan_White>
Group administrator
Mon 21 Jul 2014 04:33:20 PM UTC, comment #12: 

The Velthuis transliteration was done in the 80's when TeX did not accept more than 7bits characters; it used an analogy with the roman transliteration of Sanskrit: underdotted characters where noted with a dot before, .t, .d, etc. .r is the notation for vocalic-r

This was used with a preprocessor which converted the file with a lot of TeX macros to use the dvng fonts (still available in TeX distributions) and the mechanism to create ligatures was totally different from what is used in ttf/otf unicode fonts.

So the example you give: r.k has no meaning in this way of coding Sanskrit texts (and will be rejected by the preprocessor).

I still use this Velthuis scheme, because it is easy to write Sanskrit text with a simple keyboard and the files can be used on other OS than mine...  Moreover, with the map/tec files and xelatex, I can produce both transliterated or devanagari texts using only one file.

François Patte <frpatte>
Sun 20 Jul 2014 10:11:45 PM UTC, comment #11: 

One other point:

Your example is of using the Velthuis system.

While the Velthuis encoding r.R seems to be reasonable for this conjunct... is it really?

For consonants, such as ka, r.k would give the repha form, which in the Unicode encoding is 0x0930+0x094D+0x0915; That is, a virama is placed between the consonants. 

As I have illustrated, 0x0930+0x094D+0x090b; works in other environments.

So why is it that velthis, and these Sanskrit fonts, do something different with the vocalic r?  Why does velthuis r.R apparently produce 0x0930+0x090b, instead of 0x0930+0x094D+0x090b; ?

Steve White <Stevan_White>
Group administrator
Sun 20 Jul 2014 08:16:12 PM UTC, comment #10: 

Hi François,

Thanks for your response!  I hope you're not misunderstaning me still.

The question is not whether to form this conjunct -- the question is how to form it.  As you see from my examples, the conjunct can be formed in any of the fonts.  Only in some of the fonts, it can be formed by two different encodings.  This is what I find questionable.

The encoding 0x0930+0x094D+0x090B; looks right to me (seems conformant with the standards), and all the fonts support it.

It is still unclear to me whether the other encoding 0x0930+0x0943 that works in the Sanskrit fonts is according to standards, or perhaps some historical accident.

I can read French, but am not competent to write it!
Your image, I think, associates sounds with glyphs.
At issue here is the encoding however -- the sequence of characters occuring in a computer file -- this is not the same as the sequence of sounds of course.

Cheers!

Steve White <Stevan_White>
Group administrator
Sun 20 Jul 2014 04:16:06 PM UTC, comment #9: 

Hi Steve,

Yes I am talking for Sanskrit language.

As far as I understand what you say (my English  is not so perfect...): you have no évidence of the necessity for the ligature r+vocalic-r (U+0931+U+090b).

I searched in a few manuscripts and I must say that scribs seem to avoid the problem! Even in a printed book (ca 1950) I have, this ligature is avoided...

You need this ligature to fulfill the samdhi rules, for instance for words ending by is, us, os... followed by a word beginning by a voiced letter (and vowells are voiced letter) the "s" is changed to "r" and if the second word begins with a vocalic-r we need this ligature...

As I said above, scribs have licence to not apply samdhi rules if they want, so I cannot (today) tell you if this ligature was/is never used, but in Sanskrit grammar (at least the reference in France: Renou's Sanskrit grammar) we find a note on it (see attached image).

You maybe question Pr. Filliozat: he has a deep knowledge in Sanskrit and could give you an answer about what to think in this matter.

Best regards


François Patte <frpatte>
Sun 20 Jul 2014 02:02:26 PM UTC, comment #8: 

Hi again,

Still trying to put the pieces together here.

This seems to be an encoding question then.

THe following combinations work in all the fonts I've tried:
<p lang="sa">
ra + rvocalic : &#x0930;&#x094D;&#x090B;<br />
ra + rrvocalic : &#x0930;&#x094D;&#x0960;<br />
ra + lvocalic : &#x0930;&#x094D;&#x090C;<br />
ra + llvocalic : &#x0930;&#x094D;&#x0961;
</p>

Whereas in Siddhanta only the first two of these does something useful
<p lang="sa">
ra + rvocalic : &#x0930;&#x0943;<br />
ra + rrvocalic : &#x0930;&#x0944;<br />
la + rvocalic : &#x0932;&#x0943;<br />
la + rrvocalic : &#x0932;&#x0944;
</p>

Now in
    https://savannah.gnu.org/bugs/?39999#comment3
Shree Devi Kumar stated

"
   र + ् + ऋ = र्ऋ
   र + ृ = रृ (shows up the same as above in my default font on firefox) it should be different
"
In FreeSerif, they are different; in Siddhanta they are not.

Steve White <Stevan_White>
Group administrator
Sun 20 Jul 2014 12:11:32 PM UTC, comment #7: 

Siddhanta, FreeSerif, and Gargi agree on the encoding
     0x0930 + 0x094D + 0x090B
which seems to render the same as in your samples
     र्ऋ
See HTML file attached.

Steve White <Stevan_White>
Group administrator
Sun 20 Jul 2014 12:08:19 PM UTC, comment #6: 

See version 1.4.2 of the TDIL
SCRIPT GRAMMAR FOR HINDI LANGUAGE
   http://www.tdil-dc.in/tdildcMain/articles/737061Hindi%20Script%20Grammar%20ver%201.4-2.pdf

In the table
    Consonant and Mātrā combinations Set 3
The entry for r + r-vocalic-sign is just an X, indicating the combination cannot happen in Hindi.

So... maybe this is indeed Sanskrit-only?

Steve White <Stevan_White>
Group administrator
Sun 20 Jul 2014 10:57:05 AM UTC, comment #5: 

François, your LaTeX file specifies the language Sanskrit.

1) Of these fonts, only one, FreeSerif, responds to a language specification, makes any distinction based on language.  This makes no difference in this case however.

2) Gargi is explictly meant for rendering Hindi only.  Sanskrit 2003 and Sahadeva are explictly for rendering Sanskrit. 

There are big differences in the expected sets of conjuncts in Hindi, Sanskrit, and also in Mathili.

** François: are you referring to Sanskrit text?  Could it be that this combination only occurs in that language?

Steve White <Stevan_White>
Group administrator
Sun 20 Jul 2014 09:07:10 AM UTC, comment #4: 

Hi François,

From your response, I suspect we have miscommunicated.  This issue is not closed.

Let me repeat: I will gladly add the ligature feature, once we understand how and why it is desirable.  At this point, I have never seen why such a ligature would be used, given the existence of U+0960 and U+0961 in Unicode.

We simply havn't received the necessary documentary evidence.
We only need direct evidence of this: a standards document, or a comparison of previous standards documents.  Or perhaps some old documents that make use of an older encoding.

The ligatures are now encoded as individual characters, so it is now possible, and I suppose preferred, to use those characters.  I have checked, and found that these characters have been present since ISCII-88, so they are nothing new.

So I do not understand how it is as you said "impossible to publish a devanagari text".  Do you mean "impossible to publish certain devanagari texts without alteration"? 

You say there is a "wrong" ligature, but in your example, you only showed that no ligature is formed by FreeSerif.  This was not an oversight--it was what we thought at the time to be correct, and at this point, we still lack documentary evidence to the contrary.

Emphasizing your opinion will not help in this.  Pointing at other fonts (which seem not to agree on the point) does not help.

I am continuing to look for documents on my own, but after an hour or more of searching, no luck.  If you know of something that could help, please let us know!

Steve White <Stevan_White>
Group administrator
Sat 19 Jul 2014 08:54:55 AM UTC, comment #3: 

thank you for this answer and sorry to not have been able to find the previous notification for this bug.

I would like to add something about the "severity": it is recorded as "normal" and it might be correct regarding the whole FreeSerif font, but regarding the devanagari part it might be the highest degree, because it makes this font unusable to publish a devanagari text with this wrong ligature... but... this font is one of the nicest I know for devanagari! Pity!

Best regards

François Patte <frpatte>
Fri 18 Jul 2014 11:01:10 PM UTC, comment #2: 

Indeed this has been discussed before, but not quite resolved.

bug #39999
Devanagari - रृ is not rendered correctly

The deal is, these compounds are encoded as as single character in Unicode.  Nowadays there isn't any need for a combination, I guess.

The question is, was there a previous encoding standard wherein a character combination was used to encode these glyphs?  If so, I would be glad to add rules.  I just haven't been able to find one -- aside from some very nice fonts, but we don't regard fonts as standards.


Steve White <Stevan_White>
Group administrator
Fri 18 Jul 2014 10:37:05 PM UTC, comment #1: 

Hi François,

Thanks for this. I'll look into it -- I have a feeling I've run into it before.

Steve White <Stevan_White>
Group administrator
Fri 18 Jul 2014 03:49:05 PM UTC, original submission:  

In devanagari script, the ligature r+r vocalic (breve and long) is wrong.

This ligature should be done like the ligature r+consonant (this is an exception to the general rule for ligatures in devanagari).

I attach a pdf file (and its latex source) showing the wrong ligature given by FreeSerif and the correct ligature given by Sanskrit 2003, Nakula and Sahadeva. Gargi gives also this wrong ligature.

I find also that for devanagari, the baselineskip is not fine: in order to get a nice devanagari text, I usualy add this in my files:

\setlength{\baselineskip}{1.25\baselineskip}

Same problem with word spacing, i add this option:

WordSpace={1.5}

Regards

François Patte

François Patte <frpatte>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #31947:  semivowel-vocalics.pdf added by Stevan_White (75KiB - application/force-download)
file #31948:  klrp.tex added by Stevan_White (2KiB - text/x-tex)
file #31949:  klrp.pdf added by Stevan_White (57KiB - application/force-download)
file #31910:  semivowel-vocalics.pdf added by Stevan_White (52KiB - application/force-download)
file #31886:  R-rvocalic.tex added by Stevan_White (2KiB - text/x-tex)
file #31887:  R-rvocalic.pdf added by Stevan_White (6KiB - application/pdf)
file #31830:  test_ra_ri.pdf added by shree (61KiB - application/pdf - xelatex file and pdf showing the ra-Ri maatraa usage)
file #31829:  test_ra_ri.tex added by shree (3KiB - application/octet-stream - xelatex file and pdf showing the ra-Ri maatraa usage)
file #31750:  ligature.png added by frpatte (344KiB - image/png)
file #31738:  freetype.pdf added by frpatte (15KiB - application/pdf)
file #31739:  freetype.tex added by frpatte (1KiB - application/x-tex)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by shree (Posted a comment)
  • -email is unavailable- added by Stevan_White (Posted a comment)
  • -email is unavailable- added by frpatte (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 19 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-08-27 Stevan_White Attached File- Added Sanskrit-Renou-vocalics.png, #31977
    2014-08-24 Stevan_White Attached File- Added semivowel-vocalics.tex, #31946
        Attached File- Added semivowel-vocalics.pdf, #31947
        Attached File- Added klrp.tex, #31948
        Attached File- Added klrp.pdf, #31949
    2014-08-17 Stevan_White Attached File- Added semivowel-vocalics.pdf, #31910
        Attached File- Added semivowel-vocalics.tex, #31911
    2014-08-11 Stevan_White Attached File- Added R-rvocalic.tex, #31886
        Attached File- Added R-rvocalic.pdf, #31887
    2014-08-10 Stevan_White StatusNeed info Fix posted
        SummaryWrong devanagari ligature r+vocalic r Missing Devanagari ligature r+vocalic r for Sanskrit
    2014-08-03 shree Attached File- Added test_ra_ri.pdf, #31830
    2014-08-03 shree Attached File- Added test_ra_ri.tex, #31829
    2014-07-20 frpatte Attached File- Added ligature.png, #31750
    2014-07-18 Stevan_White StatusProceeding Need info
    2014-07-18 Stevan_White StatusNone Proceeding
        Assigned toNone Stevan_White
    2014-07-18 frpatte Attached File- Added freetype.pdf, #31738
        Attached File- Added freetype.tex, #31739

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code