wdiff - Patches: patch #7121, New, per-character diff, mode
You are not allowed to post comments on this tracker with your current authentication level.
patch #7121: New, per-character diff, mode
Submitter: | Georgios Zarkadas <gzarkadas> | ||
Submitted: | Sun 14 Mar 2010 02:05:07 PM UTC | ||
Category: | None | Priority: | 5 - Normal |
Status: | In Progress | Privacy: | Public |
Assigned to: | gagern | Open/Closed: | Open |
Tue 30 Mar 2010 08:01:42 AM UTC, comment #4: |
Martin von Gagern <gagern>![]() ![]() |
Mon 29 Mar 2010 11:04:41 PM UTC, comment #3: Hi,
|
Georgios Zarkadas <gzarkadas> |
Mon 29 Mar 2010 04:16:17 PM UTC, comment #2: OK, I've now had a second look at your patch. The general idea seems sane enough. A few remarks, in order of diff lines:
|
Martin von Gagern <gagern>![]() ![]() |
Mon 29 Mar 2010 02:08:48 PM UTC, comment #1: Thank you for your patch, Georgios!
|
Martin von Gagern <gagern>![]() ![]() |
Sun 14 Mar 2010 02:05:07 PM UTC, original submission:
I attach a diff from the 0.5 version of wdiff.c (latest stable release) that implements an additional per-character diff mode, taking into account only non-whitespace characters (just as the per-word diff mode does).
|
Georgios Zarkadas <gzarkadas> |
Depends on the following items: None found
Items that depend on this one: None found
There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.
-1- see personal e-mail.
-3-
I assume that the most GNUish way to handle different encodings would be using libiconv <http://www.gnu.org/software/libiconv/>, which is also included in glibc. That way, you can formulate your app to deal with unicode only, and rely on libiconv for input processing. Downside is that there might be encodings which have different possible encodings for the same unicode string, in which case wdiff might loose some details about the binary representation. Shouldn't worry us, though, as long as things look the same. wdiff is more about looks than bytes, I think.
Most apps don't provide special command line switches, but instead accept customization of encodings via environment variables. I'd stick with this to avoid excessive numbers of command line switches. Unless you want to control encoding for both sides of a diff independently...
By the way: I believe you want to split glyphs, not chars, i.e. you want to keep combining characters together with their base characters. This requires even more understanding of unicode, and applying it to other encodings manually would probably be very painful, so conversions to unicode feel ever more sensible.
libiconv at least works on byte buffers, not I/O streams. And it's probably slow if you pass one char at a time. Hmmm... I wonder whether we should switch from the one-char-at-a-time implementation to something processing larger buffers of bytes one at a time.
The ICU library has some documents that might be useful here:
http://userguide.icu-project.org/transforms/casemappings
http://userguide.icu-project.org/collation/concepts
That sounds great. I guess it would be useful if you could share a bzr branch of your progress. You could do so either by applying for group membership here on savannah, or using launchpad, or any other publicly accessible bzr or http server.
The only GNU thing I know about is libiconv. It does conversion, but knows little more about unicode.
The most powerful unicode library I've worked with so far is ICU, the International Components for Unicode, <http://site.icu-project.org/>. It seems pretty powerful, and also has some good documentation, to which I've already pointed you up there. One more pointer: the White_Space and Word_Break properties mentioned in http://userguide.icu-project.org/strings/properties might be useful.
ICU is no GNU project, and has over 17MiB of libraries on my setup. So it might be a useful and powerful support library, but wdiff should not depend on it, maybe not even for simple char splitting.
Looking at the ICU docs made me realize just how deep the waters are getting here. For example, if you do this correctly in a partially right-to-left script, you should ensure that unicode direction modifiers do nest properly. I assume things can become really complicated. I wouldn't expect you to address all this complexity before I consider your patch for inclusion, the word-wise diff doesn't do so either. But it might pay to think about these things, so the implementation is designed in such a way it can deal with them in the long run. Just a thought.