Thu 12 Oct 2017 10:22:44 PM UTC, comment #24:
To posterity:
David's example shows several conjunct forms each followed by a virama. These should be rendered with the virama visible -- instead they are rendered as half-form conjuncts -- no stem, no virama. This is because there were explicit table entries that mapped <conjunct full-form>+<virama> -> <conjunct half-form>, which is basically wrong. The 'half' lookup is good at determining whether a virama turns the preceding stuff into a half-form or not.
Fixed in SVN. Probably other things now broken.
|
Wed 11 Oct 2017 11:38:17 PM UTC, comment #23:
These still need fixing:
|
Wed 11 Oct 2017 07:38:16 PM UTC, comment #22:
Hi David,
I reviewed this with the latest big changes to the lookups.
Your example of 27 Aug is in the attached PDF. I can't tell anything is wrong.
Perhaps the oddest thing is the ङ्क्ष्. Velthis doesn't list a ligature for just this, but rather, in conjunction with -va, as in ङ्क्ष्व. It behaves differently depending on the text language. In Sanskrit ng-k-ssa-va forms a complex vertical ligature. In Old Hindi, then ng-k forms a vertical ligature, followed by a half-form ss and full-form va. In Hindi, I think it isn't used, but Vethuis shows a ng-virama, k-ssa half-form, va full-form.
I'm not sure about Marathi. I don't seem to have explicit tests for it, except it looks like, besides its "eyelash ra", I thought it was meant to have a set of ligatures similar to that of Old Hindi. It isn't well tested and a few ligatures are surely not forming right.
|
Fri 18 Aug 2017 12:16:50 AM UTC, comment #21:
These still need fixing:
- FreeSerif, old Hindi: ङ्क्ष्
- FreeSerif Bold, old Hindi: ङ्क्ष्
- FreeSerif Bold, Marathi: ध्न
|
Thu 17 Aug 2017 01:19:07 PM UTC, comment #20:
Hi David,
The commit I just made regarding the serif faces will likely affect the issues discussed in this report. I hope it's a good effect.
Please have a look at your tests, and tell me what you think.
|
Tue 15 Aug 2017 01:10:18 PM UTC, comment #19:
To convert a .dn file to .tex, use the program devnag.
|
Tue 15 Aug 2017 09:45:42 AM UTC, comment #18:
This problem is worse than I thouhgt.
While working on the Gujarati range, it was impressed on me that in dev2 anyway, the extent of conjuncts is determined by the 'cjct' lookups. And that effects proper placement of the i-matra at the beginning of the conjunct.
So here's a test of the Velthus table, with i-matra. Sanskrit is fine here -- Hindi, not so great.
(file #41525)
|
Tue 15 Aug 2017 09:39:01 AM UTC, comment #17:
David,
How do you latex that .dn file? I installed the required packages I think, but it chokes on the first _ character (being used I think as a virama)
|
Tue 01 Nov 2016 05:58:35 PM UTC, comment #16:
This comment thread is long, so I thought it would be helpful to list the clusters that still need fixing.
- FreeSans: ध्र् त्त् त्र्
- FreeSans Bold: त्त् त्र्
- FreeSans, Sanskrit: श्च् श्व् ञ्च् घ्न् त्न्
- FreeSans Bold, Sanskrit: श्च् श्व् घ्न् त्न्
- FreeSerif: શ્વ્ શ્ચ્
- FreeSerif, old Hindi: ङ्क्ष्
- FreeSerif Bold, old Hindi: ङ्क्ष्
- FreeSerif Bold, Marathi: ध्न
In HTML for convenient testing:
|
Thu 08 Sep 2016 04:10:30 PM UTC, comment #15:
Hi,
Pardon me, I thought you were still suggesting that I invent half forms for ligature conjucts.
Also, pardon me, I didn't pose my challenge very well (it was very late after a long day). I meant to say: in the table, find a ligature conjuct, that doesn't have a half-form, to which you can add a consonant, and still pronounce it. The reason the ligature nn has a half form is that one can follow it with a consonant to make a Sanskrit speach-sound.
I haven't attacked the Old Hindi yet. Since Old Hindi isn't a registered language, I can't specify it in 'cjct' tables. Rather I have to handle it with something triggered by something other than language, such as a 'hlig' table.
And for that to work properly with virama and following consonants, I have to make further logic tables. Kind of a mess.
And then test the whole thing again in XP. Oh dear.
|
Thu 08 Sep 2016 02:45:07 PM UTC, comment #14:
The conjunct ⟨न्न्⟩ (nn) followed by ⟨व⟩ (va) appears in the Sanskrit “यन्न्वहं”.
But I’m not sure what you mean by “the ball's in your court”: I agree with everything you’re saying about conjuncts. As far as I’m concerned the only Devanagari consonant clusters in FreeSerif that need changing are old Hindi ⟨ङ्क्ठ⟩, ⟨ङ्क्ष्⟩, etc.
|
Wed 07 Sep 2016 11:03:46 PM UTC, comment #13:
I just had a better look at your table of velthuis conjuncts.
Overall the behavior with just a virama is as we agreed it should be.
As you say, the behavior when the conjunct is followed by a consonant appears rather chaotic, and it isn't coonsitent with what I currently have.
Some of it is clear: a few combinations have half-forms (पत = pt). Many others decompose into half-forms (नन = nn). These are quite sensible.
Many more retain the ligature and put a virama beneath it. Well... the meaning is clear, but I find it rather nasty to look at. But consider for example ट्ठ्ठ. That's like tt-tth-tth. Sylvester the cat would say something like that when he was really scared. But it's not a realistic speech sound. OK this is partly because I picked the unlikely ठ to append to the conjuncts.
A consistent interpretation of this is: when the conjunct makes no sense with a following consonant, velthuis gives up, and leaves the ligature. One could as well decompose the ligature into half-forms: either way, it's not Sanskrit.
OK let's say the ball's in your court. Find me a conjunct in that table, which, followed by any consonant you like, produces a plausible speech-sound.
|
Wed 07 Sep 2016 10:26:21 PM UTC, comment #12:
For a different listing of Sanskrit conjuncts:
Charles Wilkner
A Practical Sanskrit Introductory
http://sanskritdocuments.org/learning_tutorial_wikner/wikner.pdf
p. 58
7.A.6
|
Wed 07 Sep 2016 09:35:46 PM UTC, comment #11:
Hi,
I just made a commit that resolves the problem I've been discussing, at least for the velthuis set of ligatures.
As stated: I think that (with few exceptions) the ligature conjuncts should form only if they are not followed by another consonant. I have further reason to think this is right. The modern OpenType script 'dev2' used by the Uniscribe shaping engines has a special feature 'conj' that did not exist in Windows XP.
Clearly 'conj' for conjuncts, but I hadn't understood its function before. And in fact this is just what it does: the feature is active only for conjuncts that are not followed by a further consonant.
I have only checked the behaviour for the Velthuis set of ligatures -- others are likely broken. And the whole thing is likely broken for XP and Pango.
You can see the improvement in the two test files I posted, the conjunct tests with virama and half forms. Now all the ligatures with virama just get a virama, but ligatures followed by a consonant decompose into half-forms.
|
Wed 07 Sep 2016 06:11:19 PM UTC, comment #10:
I’ve attached velthuis-conj.{dn,pdf} so you can see what Velthuis does. For each of the three styles (Sanskrit, old Hindi, and modern Hindi) the table has three columns (full form, virama form, and half form). The virama form is always the full form with a virama below the rightmost element, without exception. The half form of a conjunct is less predictable.
⟨ञ्च⟩ is a vertical ligature in Sanskrit. Its half form is also a vertical ligature, but without the stem.
⟨ट्क⟩ is a vertical ligature in Sanskrit. It has no half form, so in a half form context, it uses the virama form.
⟨क्क⟩ is a vertical ligature in Sanskrit. It has no half form, so in a half form context, it uses a horizontal sequence of two ⟨क⟩ half forms.
I don’t see any pattern. I think it is a matter of font style, not correctness. Falling back to a sequence of half forms seems reasonable.
Whatever you choose to do, don’t forget about the post-base form of ⟨य⟩, the rules for which are yet unclear to me.
https://github.com/googlei18n/noto-fonts/issues/371 lists many more Sanskrit ligatures, so you should not assume that Velthuis’s ligatures “are already representing maximal conjunctions of consonants in the Sanskrit (or Old Hindi) language”.
(file #38443, file #38444)
|
Wed 07 Sep 2016 03:52:17 PM UTC, comment #9:
Precisely what I don't know is how these conjucts are supposed to behave with a following virama, or a following virama and consonant.
I have been assured that the table in the velthuis manual is accurate. But it doesn't answer these questions.
I'm thinking... phonetically, these represent consonants pronounced without intervening vowels, as to joined half-forms in modern Hindi. It would therefore make sense to end a syllable with one of them plus a virama. For instance, in English, many words, like 'mask', end in a consonant cluster with no vowel at the end.
However, what does it mean to try to conjoin one of these ligatured cluster with another consonant without an intervening vowel? I think they are already representing maximal conjunctions of consonants in the Sanskrit (or Old Hindi) language. To stick another consonant on would be simply unpronouncable, and never occur in the language.
Unlese one is transcribing Polish into Devanagari of course. I have actually asked people about this, but I don't think I got my point across. (As I recall, some said they would just use Latin letters -- but of course Latin letters are only poorer phonetically than Indic ones.)
My judgment is -- IF another consonant should be conjoined, the system should give up on fancy ligatures and fall back to the simple half-forms. Currrently I think it does not do this.
|
Wed 07 Sep 2016 12:30:39 PM UTC, comment #8:
Do you mean that ⟨ङ्क्ष⟩ and ⟨ङ्क्ष्⟩ are meant to look identical in old Hindi?
|
Wed 07 Sep 2016 08:55:54 AM UTC, comment #7:
Hi,
I fixed the one issue with ड्र्य.
The other two things you mention, with ड्र्य and ङ्क्ष् are idosyncracies, according to the people I was working with when the Sanskrit support was implemented.
Still unsure about half forms. At least it should stop the ligature forming if it's followed by virama-consonant, I think.
|
Tue 06 Sep 2016 06:21:53 PM UTC, comment #6:
I built revision 3203 and looked at the velthuis-conj-virama.html. This bug looks solved in FreeSerif. I did notice a few things though. Velthuis has a ligature for ⟨ड्र्य⟩ in old Hindi and Sanskrit, but FreeSerif only uses the ligature for old Hindi. In old Hindi, the ⟨क्ष⟩ part of ⟨ङ्क्ष⟩ is split; this is what Velthuis does but it seems suspicious to me. ⟨ङ्क्ष्⟩ in old Hindi is missing the second virama.
For consonant ligatures without corresponding half forms, you can make up plausible half forms. For example, Sanskrit ⟨च्ञ⟩ has no half form in FreeSerif, but it is reasonable to simply remove its right vertical stroke by analogy with the half form of ⟨ञ्च⟩. Obviously, it is safer to stick with attested forms, so you could also port your HTML test files to Velthuis and see what it does.
The Indic OpenType specification mentions MANGAL.TTF as an example font. Reportedly it is bundled with VOLT (Visual OpenType Layout Tool), though I haven’t verified this myself. Maybe that font would give you some ideas on how to organize the tables.
|
Sun 04 Sep 2016 04:02:48 PM UTC, comment #5:
For the record: The tables are a mess on this point. Reason is, although modern Hindi orthography is ralatively straightforward, Sanskrit, Marathi, and Old Hindi are very idiosyncratic. And I was learning about these things as I built the tables. In the end, Devanagari is far more complicated than the other scripts.
My main reference for Sanskrit and Old Hindi was the manual of the TeX package velthuis. It lists lots of conjuncts, but doesn't mention what happens when they are followed by another consonant.
For simple cases it is certainly as we've agreed: a half-form of a ligature should appear only in combination with a following consonant.
However there are plenty of consonant ligatures in the font that don't have any corresponding half form. Maybe these would simply never occur in real text.
I'm unsure what to do with a few of them. Maybe you have some ideas on this.
The last commit of FreeSerif addresses the main problem, and corrects also some issues with half forms of conjuncts, where they were clear to me.
Please look at the attached test files. Again, my main test has long been conformance to velthuis.
Also, a perspective: I'm pretty sure I made every lookup table in FreeFont personally. Doesn't mean I knew what I was doing or I remember why.
(file #38427, file #38428, file #38429, file #38430)
|
Sat 03 Sep 2016 08:06:42 PM UTC, comment #4:
Another approach is to find clusters that work and copy whatever they do. For example, FreeSerif’s Gujarati kṣa looks right both with a virama ⟨ક્ષ્⟩ and as a half form ⟨ક્ષ્⟩. It seems to do this without any look-ahead.
|
Sat 03 Sep 2016 02:28:05 PM UTC, comment #3:
I'm amazed this hasn't been mentioned before--Devanagari has been checked by several people. Maybe these virama forms don't occur often.
One solution will be to look ahead for consonants in the lookup for half forms.
|
Sat 27 Aug 2016 03:05:49 AM UTC, comment #2:
I checked the other Indic scripts and only found two such clusters.
|
Sat 27 Aug 2016 01:59:50 AM UTC, comment #1:
This bug is more pervasive than I thought. I have found the following problematic clusters.
- FreeSans: ध्र् त्त् त्र्
- FreeSans, Sanskrit: श्च् श्व् ञ्च् घ्न् त्न्
- FreeSerif: क्ष् ज्ञ् त्त्
- FreeSerif, Sanskrit: घ्न् ञ्च् ध्न् प्त् श्च् श्व्
- FreeSerif, Marathi: ध्न
The problem is that various glyph sequences that end with virama are ligated to half forms, which is only appropriate when there is a following consonant. The Marathi cluster doesn’t end with a virama, but it’s essentially the same problem.
|
Thu 25 Aug 2016 11:00:10 PM UTC, original submission:
⟨क्ष्⟩ (U+0915 U+094D U+0937 U+094D) should look like the base form of kṣa with a virama. In FreeSerif, it looks like the half form of kṣa. The same problem applies to ⟨ज्ञ्⟩.
|