Visual Mouse Application

0 Comment

Visual Mouse Application

Vision-based user-computer interfaces involve eye-gaze pointing (Quek) and motion realization. In 1994, Istance and Howarth examined user interface interaction controlled through eye-gaze. So, Visual Mouse Application is the next generation of web application; this kind of application will help in developing a new way for the human beings to interact during any kind of online activity whether it is web surfing, online shopping or interaction with multimedia data and information streams.

The system will work by reading the eye-gaze positions as the user looks at the screen. The tracking system will be able to read and store this information. With an access to the information, like the user’s dwell time in any restricted area relative to determined thresholds, many simple tasks are conducted. This kind of processing is required to be completed in real-time. Established on certain rules, for instance specified dwell time, signals that imitate mouse clicks are sent to the computer seen by the user.


The user will sit in front of the display screen. A little infrared camera is placed under the display screen that monitors the eye-gaze of the user. The system will require just few minutes for calibration of the particular person and their posture in the tracking system.

The Control Unit of the eye-gaze tracking system gets all the data from all devices that have been connected to it.  This information is then used in calculation of eye-gaze location in vertical and horizontal axis. The Visual Mouse Application will use the stream of eye-gaze information and output the functions of the physical mouse. So, the functions of mouse on the display screen are emulated and are controlled by the user’s visual motions when he is seeing the display screen.

The applications associated with the Visual Mouse Application should have certain properties:

·         A serial port: it will allow communication with the eye-gaze tracking system.

·         Data coming from the eye-gaze tracking system should be continuously monitored.

·         The received information should be decoded in order to attain the gaze coordinates in real-time.

·         As the eye-gaze tracker is sending the coordinates of the data stream, so this information should be compared all the time.

How it works?

The Visual Mouse Application will establish a mouse click at the location where the user keeps on observing. The Visual Mouse Application will also have numerous parameters or adjustments that will enable users to set the state and mode of the program.

There will be a variable to inform that the user has moved from his earlier gaze position to another position. This variable is adjusted to “0” every time and the program gets new reference coordinates. This means it changes whenever the user is not at the earlier coordinates. When the user keeps on seeing at an area around the reference coordinates, the index variable is increased by the 1 number with every other coordinate pair located in that area. The index reference parameter has a threshold value; this is the figure the index variable has to attain in order to create the mouse click. This value can be changed; so, this will enable us to change the time required by the user to look at any specific area to create the mouse click.

Since, the index variable will be updated with every new coordinate pair and these coordinate pairs are attained at 60 Hz, so it means that the time duration before establishment of mouse click can be adjusted if we divide the index reference parameter figure by 60.

Required Software:

The Visual Mouse Application requires Microsoft Visual Studio 6.0 Enterprise Edition for its development that offers numerous utilities, whereas the recommended languages are Microsoft Visual C++ and Microsoft Foundation Class (MFC). The powerful class of EyeSerialComm can be used that includes features of ReceiveData and ReceiveStatus allowing storage and output of data. This EyeSerialComm class can  perform each major task expected by the Visual Mouse Application. This includes serial port communication, decoding of information and mouse message posting.

Eye tracking system having Visual Mouse Application will definitely be the next generation of web application (Robb) and we can hope to see this in market in coming years.

Works Cited
Quek, F. “Non-verbal vision-based interfaces”. Keynote talk, IWHIT’95 –International Workshop on Human Interface Technology ’95. Aizuwakamatsu, ukushima, Japan, 1995. <>.

Istance, H., and P. Howarth. “Keeping an eye on your interface: the potential for eye-gaze control of graphical user interfaces”. Proceedings of HCI’94. 1994. < >

Robb, Drew. “Eye Commander”. Stanford University’s EyePoint: Web Surfing With Eye Gaze. Computer World, 2007. ;;

Leave a Reply

Your email address will not be published. Required fields are marked *