Thu Jan 17 18:46:36 2008, comment #13:
I took a look at the log_accum and svnmailer, and I agree that it is not trivial at all to implement auto-guessing of the encoding; also this approach is almost certainly doomed to fail (for example, the log message in ISO-8859-15, one file in UTF-8 and another one in CP1251).
AFAIU svnmailer tries some fairly complex ways to determine the encoding, including checking svn:mime-type, the files themselves, the log message etc. It has a fallback mechanism and it's not 100% sure that the mail header will be correct. It just happens to work in most cases.
I suggest to do the following: If there is an easy way to check the body (perhaps in `build_message_body'), try if it is only US-ASCII, then UTF-8 and finally (if it is not valid), ISO-8859-1. Alternatively, just add this header (in the `send_mail' subroutine?) with UTF-8 hardcoded. That's easy and won't hurt much, I guess.
|
Mon Jan 14 20:59:06 2008, comment #12:
<<<It has
Content-Type: text/plain; charset="utf-8"
and all non-ASCII characters are readable. I can review the diff of the whole translation without any trouble.>>>
I'll have a look - but I think problems arise when content was not in UTF-8 in the first place.
Though, declaring every messages as UTF-8 could be dirty work-around that actually works better in the common case. Maybe we'll do that.
|
Mon Jan 14 20:43:42 2008, comment #11:
"I doubt it, because I don't think there's a magic solution to this issue..."
Maybe -- I don't argue at all, but I read all those mails with Mutt (takes about ~15 minutes to load on my mailbox). Basically, I only experienced this issue when I started the Bulgarian http://www.gnu.org team (and later joined webmasters) -- I receieve all non-US-ASCII messages to -unavailable- and -unavailable- as garbage, and it doesn't happen with other projects. There's a workouround, of course (Mutt and Gnus), so it is just a matter of inconvenience.
"What makes you think svnmailer and other less-sucking VCSs handle encoding correctly?"
Well, I just received this a few mins after your message:
https://www.lagaule.org/pipermail/translators/2008-January/000201.html
It has
Content-Type: text/plain; charset="utf-8"
and all non-ASCII characters are readable. I can review the diff of the whole translation without any trouble.
|
Mon Jan 14 20:11:46 2008, comment #10:
"It seems that svnmailer (and other modern VCS tools) have logic to autodetect the encoding for post-commit notifications."
I doubt it, because I don't think there's a magic solution to this issue, and because MUAs usually do not auto-detect encoding (Thunderbird does not, at least).
What makes you think svnmailer and other less-sucking VCSs handle encoding correctly? I should have a look.
|
Mon Jan 14 19:39:51 2008, comment #9:
Sylvain, I think that you can safely ignore this issue. I checked two other projects (non-Savannah) and it looks like none of them is adding a `Content-Type' header.
It seems that svnmailer (and other modern VCS tools) have logic to autodetect the encoding for post-commit notifications.
I guess it's not worth diving into is, given the fact that only old farts like me use CVS/RCS and medieval MUAs that can't autodetect encoding in the absence of this header.
Sorry for disturbing.
|
Mon Jan 14 18:15:00 2008, comment #8:
> Yavor asked for progress on this one.
As far as I'm concerned I didn't progress. We would welcome more information about how CVS deals with commit messages:
- does it add any encoding metadata? (e.g. the locale in use at commit time, if cvs is used from the command line)
- is the encoding considered ASCII?
- how well do the encoding "auto-guess" libraries can perform on such small messages?
- how practical would it be to add a charset marker at the beginning of each commit message?
As for the content of the message, this is probably more difficult - first the diff needs to be independant, for example an attachment. Then the encoding is almost certainly unknown to CVS, because any kind of text file can be versioned independently of its charset. Probably only an encoding "auto-guess" method would work.
If anyone can research about this, we use log_accum and commit_prep from the contrib/ directory in project CVS.
http://cvs.sv.gnu.org/viewvc/ccvs/contrib/?root=cvs
|
Thu Jan 18 22:54:38 2007, comment #7:
This is not an easy problem to tackle with, so we can't fix it very soon.
I had it in the tasks list:
https://savannah.gnu.org/maintenance/CvS
|
Tue Oct 3 06:40:00 2006, comment #6:
Thank you very much. Here's a link: http://zver.fsa-bg.org/pipermail/dict-notifications/2006-October/000338.html
|
Tue Oct 3 06:35:35 2006, comment #5:
I'll have a deeper look at the issue.
Can you give me a direct link to a list archive where this issue happens?
|
Tue Oct 3 06:31:17 2006, comment #4:
If this is a general setting that affects all projects, I completely agree.
But if it's possible to make this change only for "www-bg", I kindly ask you to do so. Our diffs and commit messages are always in UTF-8, so people that are using mailers that are not UTF-8-capable are doomed anyway. My request was for those who use MUAs that cannot change the encoding of the body, if it has declared (obviously wrong) encoding in the Content-Type header.
|
Mon Oct 2 19:31:08 2006, comment #3:
Some MUAs don't support UTF-8 either so I have as much chances to break things than to fix them. Therefore I'm not willing to work on this :)
|
Mon Oct 2 15:49:01 2006, comment #2:
Unfortunately it seems that the messages are coming with "Content-Type: text/plain; charset=iso-8859-1" header which make them unreadable. Some prominent MUAs like Mutt and Gnus can handle this and show them with a proper encoding, but others don't.
Could you please change the encoding to "utf-8", if it's not so hard?
|
Sat Aug 26 08:58:42 2006, comment #1:
Done. :)
|
Fri Aug 25 14:54:11 2006, original submission:
Dear Savannah hackers,
We need the commits to the Sources repository of the Bulgarian Translation Team for gnu.org to be sent as notifications by e-mail.
Project name: www-bg
Commit logs to be sent to: -unavailable-
Diffs should be included
Directories to monitor: everything
Please make the above for the Sources repository only.
Thanks in advance and happy hacking.
|