(r20) DuckyThesisProposalNotes < Main

Tags: view all tags
---+ Ducky Thesis Proposal Notes
%TOC%

---++ Problem Statement

I propose to study which techniques developers use when developing code, how that varies from person to person, and how success at programming tasks correlates with choice of techniques.  

I will do so via a user study that captures information about several assigned tasks:
   * the questions that developers ask about the code for that task
   * which IDE interactions they do to answer them
   * the source code that they add/modify/delete
   * the time that it takes to do the task
   * the quality of their code, as measured by the number of unit tests which pass
From that, I will examine which techniques they use to answer which questions, and correlate the choice of technique with how fast and how well they complete the task.


---+++ Problems:
   * P1: We (software engineering researchers) do not know what different low-level techniques developers use when developing code using an IDE.
      * P1.1: We do not have a shared vocabulary for discussing different techniques that developers use when developing code with IDEs. (?)
   * P2: We do not know which techniques are the most productive.
   * P3: We do not know how to teach/train developers how to be more productive.
   * P4: Coding video is expensive, annoying, and it is difficult to be consistent in coding (especially when one doesn't know what one is looking for)

---+++ Givens:

Jonathan Sillito showed:
   * G1: Questions that developers ask in the pursuit of code can be classified.

---+++ Hypotheses:
   * H0.0: We can design a study that will force people ask (generally) the same questions.
   * H0.1: We can figure out what questions they are asking.
   * H0.2: When guiding people to ask difficult questions, they will also by 
   * H1: There are a few (and only a few) different techniques for pulling answers to a given question out of a code base via an IDE.
   * H2: Which technique people use to pull information out of the source code via the IDE correlates with how fast they can at answering the question.
   * H3: Which technique people use to pull information out of the source code via the IDE correlates with the quality of their solution.
   * H4: We can figure out when people get stuck by the patterns of their interactions with the IDE.



---++ Literature Review
   * Murphy/Kersten/Findlater
   * Robillard et al
   * BSD et al (unpublished)
   * Jonathan Sillito's questions
   * Andrew Ko ISCE05 
   * (programmer variability studies: Sackman/Erikson/Grant, Dickey, Curtis, <nop>DeMarco and Lister)

---++ Proposed data-gathering methods

10-15 pairs of programmers will be videotaped using Eclipse to perform three to five different (small) programming assignments.  Their interactions with Eclipse will be logged using the Mylar Monitor.  

The tasks will be designed to attempt to force the users to ask specific, relatively difficult types of questions.  We expect that they will also ask a number of "easier" questions in the course of answering the more complex questions.

Prior to the large study, I will have casual discussions with a number of my collaborators in the Software Practices Lab to assess likely candidates for tasks that will elicit specific questions.

I will run pilots of the study with collaborators in Software Practices and HCI.  Three pilots will be talk-aloud done by individual collaborators (who are sophisticated enough to give good talk-aloud data).   Two will be pair-programming.  In all five cases, I will do a semi-structured interview with the participants to gauge the effectiveness of the study.

I will then run three pilots with people who are not familiar with this project and its goals.


---++ Proposed analysis methods

I will watch the video specifically to look for which questions people ask, and I will put those questions into Sillito's taxonomy.

I will examine the Mylar Monitor logs "by hand" to see which techniques people use to answer the questions that arise.  Both successful techniques -- those which lead to an answer to the question -- and unsuccessful techniques will be noted.

I will also note points where the participants are "stuck".

I will also use data mining/machine learning techniques to find patterns that correlate with 
   * speed
   * code quality
   * being "stuck"


   * Set ALLOWTOPICCHANGE = DuckySherwood, GailMurphy
Topic revision: r20 - 2007-03-02 - DuckySherwood
~~Edit~~
~~Attach~~