LibStylus Design Document

Tomas Frydrych <tomasfrydrych yahoo co uk>

Revision History

10 November 2005: Initial version.

Rationale

One of the challenges for a hand-held device is optimatisation of user input in the absence of traditional keyboard and mouse. A possible and common answer is to provide a presure sensitive screen and a stylus. While on the most basic level in this scenario the stylus replaces the mouse, typically it will be accompanied by a virtual (on screen) keyboard and handwriting recognition software that allows it to be used in the place of a keyboard. However, both the use of virtual keyboard and of handwriting recognition can be tedious in real life, particularly where the screen is small.

At the same time, the stylus is not a mouse, but rather a pen and there are situations where pen-like input is much more natural than using a keyboard, and where an emulation of the oldfashioned paper-ink model opens new possibilites for use; this is where electronic ink comes in: the idea is to allow the user to scribble into existing documents using their stylus (for some illustrations, see the home page of TAbiWord).

Conceptually the idea of using electronic ink in an application is simple but its implementation requires certain amount of low-level processing that would need to be duplicated in each application. The aim of libstylus is to provide an API that would allow an application to implement ink support without having to deal with the low level issues.

Overal Design Considerations

Preliminaries

•  The code should be written so as to make it adapdable to different application kits (gtk, Qt, etc.).
•  Since the library is being developed alongside AbiWord to provided in the first instance ink support in AbiWord, it will be written in C++, but C bindings will be provided to make it widely usable.
• Two versions of the library, one with floating point api and one with integer api will be availabe.

Needed API

Input modes

•  Manual mode (application tells us when to interprete stylus activity as ink, when to ignore it).
•  Automatic mode (we decide if given stylus activity is to be translated to ink or passed down to the application for further processing).

An object to encapsulate ink

• This object should be entirely opaque to the application.

API to transform ink objects

• Scale ink in x and y axes,
• Split ink vertically, horizontally and into quadrants,
• Merge two or more ink objects into one.

API to query ink properties

• Test whether ink area is enclosed or open,
• Test whether a point is located inside enclosed ink area,
• Test whether, given ascent, descent and baseline origin, the ink represents underlining,
• Test whether, given ascent, descent and baseline origin, the ink represents striking-out.

API to set ink properties

• Set colour,
• Set line style.

API to draw ink object

• Draw ink object at given coordinances and with a given zoom factor (the draw must only modify the screen in the actual ink pixels, so that it could be laid over existing screen content),
• Clear screen area associated with the ink,
• Test whether given rectangle intersects the ink.

API to associate application-specific data with the object

• Association of data with the origin and extents of the object (to facilitate maintaining of correct relationship between ink and the underlying document.

API to serialise ink

• Serialise into InkML
• Create ink object from InkML

API to facilitate conversion of ink to other media

• Ink to text conversion.

Basic Design

The library will be of modular design consisting of a frontend and a backend.

LibStylus FrontEnd

•  The front end will provide an STY_Ink object and methods to manipulate, transform, draw, serialise and deserialise the object.
• The front end will be written entirely in ANSI C++ and have as few extra dependencies as possible (probably none).

LibStylus BackEnds

The library will contain one or more backends, each of which will provide two sets of services to the front end:

• Capture of stylus activity and it's translation into STY_Objects,
• Drawing on screen.

The functionality of the backend is encapsulated by an abstract class STY_Backend; each backend will implement a derrived class of it.

The backends are managed by STY_BackendManager, a singleton class that facilitates the creation of required backend instances by the application.

Current plan is to provide a backend using the AbiWord application framework and a generic gtk backend. A Qt based backend would be desirable (contributors are welcome). (Should it be desirable, the application can provide also it's own backend.)