A Redesign of the Results Section of Study One in Studying the Language and Structure in Non-Programmers' Solutions to Programming Problems by Pane, Ratanamahatana, and Myers

Programming Style

The raters classified each statement or sentence in the solutions into one of the following categories based on the style of programming that it most closely matches.

Prevalence: 22.7 occurrences per participant.

Construct	Occurrences	Example
production rules or event-based, beginning with when, if, or after.	54%	When PacMan eats all the dots, he goes to the next level.
constraints, where relations are stated which should always hold.	18%	PacMan cannot go through a wall.
other (98% of these were classified by the raters as declarative statements).	16%	There are 4 monsters.
imperative, where a sequence of commands is specified.	12%	Start with this image. Play this sound. Display "Player One Get Ready."

Perspective

Beginners sometimes confuse their role or perspective while they are developing a program. Instead of thinking about the program from the perspective of the programmer, they might adopt the role of the end-user of the program, or in the case of games and stories, one of the characters portrayed by the program. The raters classified the participants' statements according to the perspective or role that they indicated.

Prevalence: 23.2 occurrences per participant

Construct	Occurrences	Example
player's or end-user's perspective.	45%	When I push the left arrow PacMan goes left.
programmer's perspective.	34%	If arrow for Player 1 is "left" move PacMan left.
other (99% of these were classified by the raters as third-person perspective).	20%	If he eats a power pill and he eats the ghosts, they will die.

Modifying State

The raters examined places where the participants were making changes to an entity.

Prevalence: 4.6 occurrences per participant.

Construct	Occurrences	Example
behaviors were built into the entity, in an object-oriented fashion.	61%	Get the big dot and the ghost will turn colors...
direct modification of the properties of entities.	20%	After eating a large dot, change the ghosts from original color to blue.
other.	18%

Pictures

In addition to the above classifications done by the raters, the experimenter examined each solution to determine whether pictures were drawn as part of the solution.

Construct	Occurrences	Example
included at least one picture.	67%
used text only.	33%

AND

The raters examined the intended meaning when the participants used the word AND.

Prevalence: 6.3 occurrences per participant.

Construct	Occurrences	Example
boolean conjunction.	67%	If PacMan is travelling up and hits a wall, the player should...
for sequencing, to mean next or afterward.	29%	PacMan eats a big blinking dot, and then the ghosts turn blue.
other	3%	Every level the fruit should stay for less and less seconds.

OR

The raters examined the intended meaning when the participants used the word OR.

Prevalence: 1.5 occurrences per participant.

Construct	Occurrences	Example
boolean disjunction.	63%	To make PacMan go up or down, you push the up or down arrow key.
clarifying or restating the prior item.	24%	When PacMan hits a ghost or a monster, he loses his life.
meaning otherwise	8%
other.	5%

THEN

The raters examined the intended meaning when the participants used the word THEN.

Prevalence: 2.2 occurrences per participant.

Construct	Occurrences	Example
sequencing, to mean next or afterward.	66%	First he eats the fruit, then his score goes up 100 points.
meaning consequently, or in that case.	32%	If you eat all the dots then you go to a higher level.
to mean besides or also.	1%
other.	1%

Operations on Multiple Objects

The raters examined those statements that operate on multiple objects, where some or all of the objects are affected by the operation.

Prevalence: 6.1 occurrences per participant.

Construct	Occurrences	Example
set and subset specifications.	95%	When PacMan gets all the dots, he goes to the next level.
loops or iteration.	5%	#5 moves down to #6, #6 moves to #7, etc. until #10 which is kicked off the high score list.

Iteration or Looping Constructs

The raters examined those statements that were either implicit or explicit looping constructs.

Prevalence: 1.6 occurrences per participant.

Construct	Occurrences	Example
implicit, where only a terminating condition is specified.	73%	Make PacMan go left until a dead end.
explicit, with keywords such as repeat, while, and so on, etc.	20%
other.	7%

Complex Conditionals

The raters examined those statements that specify conditions with multiple options.

Prevalence: 2.3 occurrences per participant.

Construct	Occurrences	Example
a set of mutually exclusive rules.	37%	When the monster is green he can kill PacMan. When the monster is blue PacMan can eat the monster.
a general condition, subsequently modified with exceptions.	27%	When you encounter a ghost, the ghost should kill you. But if you have a power pill you can eat them.
boolean expressions.	23%	After eating a blinking dot and eating a blue and blinking ghost, he should get points.
other (95% of these either listed the exception first, or did not list a general case).	14%	If he gets a [power pill] then if you run into them you get points.

Remembering State

The raters examined the methods used to keep track of state when an action in the past should affect a subsequent action.

Prevalence: 4.1 occurrences per participant.

Construct	Occurrences	Example
using present tense when mentioning the past event.	56%	When PacMan eats a special dot he is able to eat the ghosts.
using the word after.	19%	After using up the power pill, the ghosts can eat PacMan again.
using a state variable to track information about the past event.	11%	When the monster is blue PacMan can eat the monster.
mentioning the future event at the time of past event.	6%	When PacMan gets a shiny dot, then if you run into the ghosts, you get points.
using the past tense when mentioning the past event.	5%	In about 10 seconds, if PacMan didn't eat it take it off again.
other.	4%

Tracking Progress

The raters examined the methods used to keep track of progress through a long task.

Prevalence: 2.0 occurrences per participant.

Construct	Occurrences	Example
all or nothing, where tracking is implicit or done with sets.	85%	When PacMan gets all the dots, he goes to the next level.
using counting, where a variable such as a counter tracks the progress.	14%	When PacMan loses 3 lives, it's game over.
other.	1%

Mathematical Operations

The raters examined the kinds of notations used to specify mathematical operations.

Prevalence: 3.4 occurrences per participant.

Construct	Occurrences	Example
natural language style, missing the amount or the variable.	59%	When he eats the pill, he gets more points...
natural language style, with no missing information.	40%	When PacMan eats a big dot, add 100 points to the score.
programming language style (count = count + 20)	0%
mathematical style (count + 20)	0%

Motions

The raters examined the participants' expectations about whether motions of objects should require explicit incremental updating.

Prevalence: 7.8 occurrences per participant.

Construct	Occurrences	Example
expect continuous motion, specifying only changes in motion.	97%	PacMan stops when he hits a wall.
continually update the positions of moving objects.	2%
other.	1%

Randomness

The raters examined the methods used by the participants in expressing events that were supposed to happen at uncertain times or with uncertain durations.

Prevalence: 1.4 occurrences per participant.

Construct	Occurrences	Example
using precision, where no element of uncertainty is expressed.	47%	Put the new fruit in every 30 seconds.
using words other than random to express the uncertainty.	20%	The fruit will go away after a while.
using precision with hedging to express uncertainty.	18%	After around 3 or 4 more seconds the fruit disappears.
other (often the action was tied to another event).	15%	Put a fruit on the screen when PacMan is running out of power.
used the word random.	0%

Insertion into a Data Structure

The raters examined the methods used by the participants to insert an element into the middle of an existing sequence of elements.

Prevalence: 1.0 occurrences per participant.

Construct	Occurrences	Example
inserting first, repositioning other elements afterwards.	48%
no mention of making room for the inserted element.	26%
making space by repositioning others, then inserting the element.	17%
other.	8%

Sorted Insertion

The raters examined the methods used by the participants to determine the correct place to insert an element into a sorted list.

Prevalence: 1.1 occurrences per participant.

Construct	Occurrences	Example
using an incorrect method, with missing or incorrect details.	43%
a method that is correct for the current data, but not a correct general solution.	28%
a correct general method that would work for any data.	18%
other	10%