bugXnee - Bugs: bug #28238, XI2 extension support

 
 

bug #28238: XI2 extension support

Submitted by:  Rami Ylimaki <rjy>
Submitted on:  Wed 09 Dec 2009 10:43:24 AM UTC  
 
Category: libxneeSeverity: 3 - Normal
Item Group: Feature requestStatus: Fixed
Privacy: PublicAssigned to: Henrik Sandklef <hesa>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sat 19 Feb 2011 02:55:08 PM UTC, comment #9:

Available both in CVS
and in last release.

Henrik Sandklef <hesa>
Project AdministratorIn charge of this item.
Wed 26 May 2010 09:32:28 AM UTC, comment #8:

It's kind of working. Early adopters go here:

http://itupw056.itu.chalmers.se/xnee/special-dist/xinput/xnee-3.05-xinput-0.1.tar.gz

Henrik Sandklef <hesa>
Project AdministratorIn charge of this item.
Fri 16 Apr 2010 09:16:40 PM UTC, comment #7:

I am working on this bug again, just wanted to update you

Henrik Sandklef <hesa>
Project AdministratorIn charge of this item.
Sun 24 Jan 2010 06:02:55 PM UTC, comment #6:

Adding support for multi devices to swinput.
Will work in Xnee after that

Henrik Sandklef <hesa>
Project AdministratorIn charge of this item.
Mon 11 Jan 2010 12:53:10 PM UTC, comment #5:

Thanks for looking into this.

I'll warn you about a potential problem in X server. The record extension was disabled a while back in X server code because it didn't work well after XI2 changes. This has been fixed, but as far as I know, the patch hasn't been committed to the official X server tree yet. This means that there probably isn't recording support even in debian experimental.

The patch is still in Peter's private X server tree: http://cgit.freedesktop.org/~whot/xserver/log/?h=record but I didn't see it in the official tree. He will probably commit it soon after he gets more testing feedback: http://lists.freedesktop.org/archives/xorg-devel/2010-January/004836.html.

Rami Ylimaki <rjy>
Sat 09 Jan 2010 02:37:10 PM UTC, comment #4:

Using a separate branch for this:
xi2-support

Henrik Sandklef <hesa>
Project AdministratorIn charge of this item.
Sat 09 Jan 2010 02:35:43 PM UTC, comment #3:

Have found a spare computer. Will install Debian GNU/Lenny (experimental) and start looking at this.

Henrik Sandklef <hesa>
Project AdministratorIn charge of this item.
Thu 10 Dec 2009 10:23:06 AM UTC, comment #2:

Below is a more detailed description of what I'd like CNEE to be able to do. Please clarify if I have misunderstood something.

1. Current situation: recording and playback of core input events

Let's assume that I have a simple drawing application and I'll use CNEE to test it. Here are the steps that I'd take:

a. start recording pointer events: "cnee --record --mouse > output.txt"
b. draw to the application window
c. stop recording by terminating cnee

As a result the output.txt contains lines like:

0,6,228,181,0,0,4294967295,156381343
0,4,0,0,1,0,0,156381344
0,6,228,180,0,0,4294967295,156381361
...
0,5,0,0,1,0,0,156382915

CNEE has decoded the information from MotionNotify (coordinates are there), ButtonPress and ButtonRelease events and saved them to output.txt.

Sometime after recording and saving output.txt, I'd playback these events to see that the drawing application is still producing desired results:

a. execute "cnee --replay -f output.txt"
b. examine the drawing application canvas
c. a familiar figure appears on the screen

2. Feature request: recording and playback of extension input events

If the drawing application is modernized to support extension events, I can't test the application anymore.

a. start recording: "cnee --record --devera 2-80 > output.txt
b. draw
c. stop recording

I can see the extension events now, but they aren't decoded anywhere. In output.txt, I'll see the following kind of lines:
...
0,73,0,0,0,0,0,156935358
...

There's no information about coordinates or pressure levels on those lines, so I won't be able to replicate the drawing using extension events.

My understanding is that the following modifications should be done to CNEE to make this work:
a. Check if X server supports XI2
b. If so, decode DeviceMotionNotify, DeviceButtonPress and DeviceButtonRelease in xnee_record_handle_event_printer just like MotionNotify, ButtonPress and ButtonRelease are decoded.
c. When replaying output.txt, the core motion events are faked in xnee_fake_motion_event with XTestFakeMotionNotify. You'd have to call XTestFakeDeviceMotionNotify for the extension motion events. Similarly two different versions of the faking functions can be found for the button presses and releases.

3. Limiting recording to specified devices

After implementing the above changes, CNEE would be able to record and replay all input events. The core X protocol is aware of only one pointer and one keyboard. However, XI2 allows the user to have multiple devices. Therefore the decoding of extension events would have to include information about the device id that originated the events so that the events can be played back for correct devices.

By default CNEE could record from all devices if --mouse option is given. It would be nice if the user could limit recording to some specific device. In this case the user would use --mouse="my touchpad" or --mouse="my bluetooth mouse" to limit recording to a specified device.

Also it would be nice if the user could change the input device on playback. The user would first create a test case with --mouse="my bluetooth mouse". Later the configuration on his computer has changed but he still wants to replay the test case. In this case the --replay method would need to have some kind of option to tell that events originating from "my bluetooth mouse" should be played back from "my usb mouse".

Before implementing this kind of functionality, it'd would be easy to just assume everything originates from "Virtual core pointer" and play the events back for that device also. That would probably work correctly on most use cases because usually the pointer tests are done for a single pointer. The important thing would be to allow recording and playback of extension events for the XI2 default "Virtual core pointer" and worry about specific devices later.

4. Keyboards

I'm talking about pointers above because that's what I'm currently mostly interested in. However, everything that has been said above applies to --keyboard option and KeyPress/KeyRelease DeviceKeyPress/DeviceKeyRelease.

5. Event ranges

As I have understood it, CNEE just records whatever event IDs the X server is sending. However, to identify extension events, X server uses a base event number for the extension and the actual event numbers are relative to the base. When recording extension input events, CNEE can't just save an event number like 74 to mean DeviceButtonRelease, because the event base may be a different number on different systems. Maybe CNEE should just print names for the input events when they are recorded or be a little more generic and print "XI2+1" instead of "74" for the DeviceButtonRelease.

This kind of functionality is needed if the user wants to record a test case on one computer and then play it back later on a different computer. The current legacy functionality doesn't have this problem, because the core input event IDs are pretty much hard coded to known values.

Rami Ylimaki <rjy>
Wed 09 Dec 2009 05:31:03 PM UTC, comment #1:

hi
.. and thanks for your feature request.

Xnee can record all events, including extension events.

As far as replaying I will have a look at it.

/hesa

... having seen a presentation on gtk and XI2 at FSCONS (fscons.org) I do want to check this out :)

Henrik Sandklef <hesa>
Project AdministratorIn charge of this item.
Wed 09 Dec 2009 10:43:24 AM UTC, original submission:

CNEE would be a very powerful tool if it was updated to support the XI2 extension. Currently it is only able to record and playback core events. This functionality should be extended so that the user can give an XI2 input device id or name as an argument and this would make CNEE to record or playback events only from the specified device.

On the recording side I'd like to see the extension events handled in addition to ṕlain core events (DeviceMotionNotify/MotionNotify and similar events for button and key presses).

On the playback side the XTest extension already supports extension events and those should be played back instead of core events (XTestFakeDeviceMotionEvent/XTestFakeMotionEvent and similar calls for button and key presses) if the user chose to record some particular input device.

Rami Ylimaki <rjy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by hesa (Posted a comment)
  • -unavailable- added by rjy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 19 Feb 2011 08:49:14 PM UTChesaOpen/ClosedOpen=>Closed
    Sat 19 Feb 2011 02:55:08 PM UTChesaStatusIn Progress=>Fixed
    Wed 09 Dec 2009 05:31:03 PM UTChesaStatusNone=>In Progress
      Assigned toNone=>hesa

    Back to the top


    Powered by Savane 3.1-cleanup1