Difference: BikramAdhikariProjectIter3 (1 vs. 7)

Iteration #3 : A technical perspective

Speed Control: In this policy, the wizard restricts maximum magnitude of control signal based on proximity of nearby obstacles.

Direction Control: In this policy, the wizard takes control of the steering if the PWC crosses a threshold distance from an obstacle, steers the PWC to the nearest free space and releases control back to the user. The user has control of magnitude of speed at all times.

Autonomous Driving: In this policy, the wizard takes full control of the PWC to perform a task.

In the following section, we describe an example scenario showing the first two control policies in action from our previous work in [16]. We highlight the limitations of the control policies and describe our proposed intermediate control policy.

Fig[1] An example scenario of parking at a table task

Given a parking task at a table, the PWC user is required to manoeuvre the PWC to park at the table while avoiding possible obstacles on the way (see Fig. 1). The following plots describe shared control executed using combination of participant, wizard and effective shared control. The values plotted are polar magnitude and angle of joystick which maps to wheelchair motion. The top plot shows the magnitude of joystick position in polar coordinate that represents speed of the PWC and bottom plot show the direction (theta) of the joystick that represents the steering of the PWC.

In speed control policy (see Fig. 2), we observe that the steering (theta) from the user follows the shared controlled trajectory as long as the wizard limiting speed is non-zero. When the wizard's limiting speed was zero (eg. 40sec - 45 sec), the shared steering does not follow participant steering. During these conditions, the user experienced lack of guidance from the system for suitable obstacle free navigation.

Fig[2] Shared Control using speed control (left: occupancy map showing PWC trajectory; right: time series plot showing control signals)

In direction control policy (see Fig. 2), we observe that the steering (theta) from the user follows the shared controlled trajectory when the teleoperator is not overriding the user's joystick. During override, the user's joystick motion is actually pointing in the reverse direction of actual shared steering of the PWC (eg. 15 sec - 22 sec and 30 sec - 40 sec). During the transition of steering control from PWC user to the teleoperator and vice versa we observed a discontinuity similar to one described in [2]. This discontinuity led to user getting confused about their expectation from joystick.

Fig[3] Shared Control using direction control (left: occupancy map showing PWC trajectory; right: time series plot showing control signals)

Wizard of Oz interfaces have drawn interest in research community as a tool to train novice elderly PWC users. Problems associated with unintuitive guidance (in speed control policy) and switching discontinuity (in direction control) make it difficult for novice elderly PWC user to comprehend PWC behavior. In this project, we intend to bridge this discontinuity in direction control policy and lack of direction guidance in speed control policy. We integrate, speed control with steering guidance using shear force on a custom designed low cost joystick handle. This form of guidance has not been tested before. We hypothesize that, our proposed shared control strategy will keep the user in control at all times and provide assistance when the user is unclear about suitable driving direction.

Our first and second design iterations focused on exploring the problem space and possible technical solution space. We leave the discussion on findings from the iteration for final report. In the next section, we describe an experiment we propose to conduct to test the

[14] Q. Li, W. Chen, and J. Wang, “Dynamic shared control for human-wheelchair cooperation,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), 2011, pp. 4278–4283.

[15] P. Viswanathan, R. Wang, and A. Mihailidis, “Wizard-of-Oz and mixed-methods studies to inform intelligent wheelchair design for older adults with dementia,” in Association for the Advancement of Assistive Technology in Europe, 2013.

[16] I. M. Mitchell, P. Viswanathan, B. Adhikari, E. Rothfels and A. K. Mackworth, "Shared control policies for safe wheelchair navigation of elderly adults with cognitive and mobility impairments: designing a Wizard of oz study", in American Control Conference, 2014.

Shared control have been used in applications involving remote operation such as surgery and pilot training systems which assume users to be trained professionals. Literature in shared control for lane assistance for passenger cars appear more closely related to our application as these systems also assume users to be novice drivers and the degrees of freedom of motion is also restricted to a plane. [2] describes three approaches to shared road departure prevention (RDP) for simulated emergency manoeuvre. They experimented haptic feedback (HF), drive by wire (DBW) and combination of HF and DBW with normal driving. In HF, given a likelihood of a road departure, the RDP applied an advisory steering torque such that the two agents would carry out the emergency manoeuvre cooperatively. In DBW, given a likelihood of road departure, the RDP adjusted the front-wheels angle to keep the vehicle on the road. In this mode, the users steering signal is completely overridden by the RDP. Their experiments on 30 participants in a vehicle simulator suggested that HF had no significant effect on the vehicle's path or the likelihood of road departure.The authors describe that, users perceived strong haptic feedback was authoritarian where as they generated more torque on the steering wheel against HF and overrode the RDP system if the HF was not strong. DBW and DBW+HF reduced the likelihood of departure. However the authors report degraded stimulus-response compatibility with DBW systems because when the DBW system took over control, the users would not steering wheel turning, which lead to confuse their internal perception of the vehicle.

Taking inspiration from their work, our system uses an approach similar to DBW. Force-feedback haptic joysticks have been used in smart PWCs [3,4] but they are bulky, expensive and/or lack sufficient torque. Vibration feedback on seat [5] and steering wheel [6] have shown to have positive effect on performance,learning of a lane keeping task and reduction in reaction time and frontal collision. Without totally discarding haptic channel, our previous study used simple vibration actuator mounted below the joystick to inform user if the joystick signal is being modified or not. However, it was not suggestive for users regarding the direction they would have to move in case of an obstacle on their way. We propose to render shear force on palm of users driving hand as advisory direction guidance under these circumstances.

Collaborative wheelchair assistant (CWA) [9],[10] modified users input by computing motion perpendicular to the desired path. Perpendicular motion away from the desired path increased the elastic path controllers output and force the PWC to return to the path. The amount of guidance was determined by varying a parameter that controlled the elastic path controllers gain. Collaborative wheelchair control (CWC) [3] used smoothness, directness and safety measures to compute local efficiencies of human and robot control signals. The system blended the control signals based on current and past average relative efficiencies.

It is difficult to recruit a large number of wheelchair users on the target demographics which makes it difficult to produce statistically significant results [11],[12]. As [10] and [11], we will use able-bodied users to evaluate the performance of our system. Future work will include a case study with an user from the target population.

[3] C. Urdiales, J. Peula, M. Fernandez-Carmona, C. Barru ́ , E. P ́ rez,S ́ nchez-Tato, J. del Toro, F. Galluppi, U. Cort ́ s, R. Annichiaricco,C. Caltagirone, and F. Sandoval, “A new multi-criteria optimization strategy for shared control in wheelchair assisted navigation,” Autonomous Robots, vol. 30, no. 2, pp. 179–197, 2011.

[4] E. B. Vander Poorten, E. Demeester, E. Reekmans, J. Philips, A. Huntemann, and J. De Schutter, “Powered wheelchair navigation assistance through kinematically correct environmental haptic feedback,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), 2012, pp. 3706–3712.

[5] S. de Groot, J. C. F. de Winter, J. M. L ́ pez Garc ́a, M. Mulder,and P. A. Wieringa, “The effect of concurrent bandwidth feedback on learning the lane-keeping task in a driving simulator.” Human Factors,vol. 53, no. 1, pp. 50 – 62, 2011.

[6] J. Chun, S. H. Han, G. Park, J. Seo, I. Lee, and S. Choi, “Evaluation of vibrotactile feedback for forward collision warning on the steering wheel and seatbelt,” Int. Journal of Industrial Ergonomics, vol. 42, no. 5, pp. 443 – 448, 2012.

[7] R. C. Simpson, “Smart wheelchairs: A literature review,” Journal of Rehabilitation Research and Development, vol. 42, no. 4, pp. 423–438, 2005.

[8] P. Viswanathan, J. J. Little, A. K. Mackworth, and A. Mihailidis, “Navigation and obstacle avoidance help (NOAH) for older adults with cognitive impairment: a pilot study,” in Proc. Int. ACM SIGACCESS Conf. on Computers and Accessibility (ASSETS), 2011, pp. 43–50.

[9] Q. Zeng, C. L. Teo, B. Rebsamen, and E. Burdet, “A collaborative wheelchair system,” IEEE Trans. Neural Systems and Rehabilitation Engineering, vol. 16, no. 2, pp. 161–170, 2008.

[10] Q. Zeng, E. Burdet, and C. L. Teo, “Evaluation of a collaborative wheelchair system in cerebral palsy and traumatic brain injury users,” Neurorehabilitation and Neural Repair, vol. 23, no. 5, pp. 494–504, 2009.

[11] T. Carlson and Y. Demiris, “Collaborative control for a robotic wheelchair: evaluation of performance, attention, and workload,” IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics,, vol. 42, no. 3, pp. 876–888, 2012.

[12] H. A. Yanco, “Shared user-computer control of a robotic wheelchair system,” Ph.D. dissertation, MIT, Cambridge, MA, 2000.

CPSC 543 Project - Third Iteration

In this iteration, we justify our shared control approach in context of existing literature in shared control of passenger cars and smart Powered Wheelchair (PWC) research with shared control that has been tested on users with cognitive and/or mobility impairments. Following it, we describe our current progress towards experimental setup and experiment design.

In this iteration, We use ideas about possible shapes and size of the joystick interface from previous iteration. Circular egocentric view around the wheelchair joystick can be realized using combination of visual percept and tactile sub-addition. We describe the exploratory technology implementation phase of our physical user interface design project in the following section.

Shared Control

Shared Control in PWC

Our contribution

Experimental Setup

Egocentric View

Visual Display

Figure [1] Egocentric LED display around wheelchair joystick

Our first attempt was to use egocentric LED display to point in the direction towards the which the intelligent system wants to guide. Figure [2] shows the brightest green as the direction of possible heading.

Figure [2] LED display showing green head in the direction away from obstacle

Another form of display would be suggestive of direction towards which the joystick could turn. We simulated this using trailing LEDs. This video shows the green trail turning in clockwise direction. We could use this form of display if we need direction only visual guidance. We have not experimented with multiple colors and other patterns. However above two forms cover position and gradient component of guidance using visual feedback.

Haptic Rendering

video

This method has its challenges mainly because the motor we are using is not strong enough to produce any torque when the palm solidly rests on it. Using a bigger motor would be another option which we will explore in the next iteration.

For now, we could possibly not bring the motor in touch with user's hand surface but generate a vibration either to represent position or direction.This video shows a vibration motor turning round inside the surface of the hemispherical ball. The vibration from the motor produced was hard to localize spatially. This could be due to the hollow space and light surface material made of styrofoam. To reduce the transfer of vibration from the motor to the servo motor shaft, the vibration motor was orthogonally attached to a loosely coupled plastic surface. Attaching the vibration motor orthogonal to the rotary plane of the servo motor reduced unnecessary vibration significantly.

To make the vibration localized within a region, the vibration motor was made to touch the inner surface of the surrounding inner wall around the joystick. This transferred the vibration on to that surface. On removing some more material from the styrofoam on the sides, we were able to perceive change in vibration as the serve shaft turned around the joystick.This video shows the mounting of vibration motor and we test if the vibration was perceived any better. However, we could only perceive slight change in position of vibration.

As vibration was hard to localize, we removed vibration at the end of the servo motor shaft and replaced it with an eccentric wheel. This eccentric wheel protruded right enough outside the surface of the joystick to bring a sensation of guiding movement. This example video shows how the eccentric wheel produce shear force on fingers of an user.

Egocentric Input

video

Figure [3]: Touch sensors (circular and point pressure type touch sensors)used in this project

Reflections

The input section experimented in this iteration is merely an illustration of a working sensory system. We intend to use this sensory input to control the pan-tilt of the wheelchair camera so that the user can obtain vision based system's assistance. Right now, we have only used the circular touch sensor. This sensor could be used to turn the camera around by mapping sensor to the camera position (similar to mapping LED display with sensor position). We intend to incorporate controllability of tilt as well using combination of circular and point touch sensor with gestures such as swiping up and down to tilt the camera up and down.

Next Iteration

Reference:

[1] Wang, R. H., Mihailidis, A., Dutta, T. and Fernie, G. R. (2011). Usability testing of multimodal feedback interface and simulated collision-avoidance power wheelchair for long-term-care home residents with cognitive impairments. Journal of Rehabilitation Research and Development, 48(6), 801-22. doi:10.1682/JRRD.2010.08.0147

-- BikramAdhikari - 24 Mar 2014