Participant 1

Initial description: Change the color of two objects to match the color of the objects of a similar shape that are near the bottom of the example. Then make the same change but start from the objects near the top of the example. Then repeat.

Final description: Change the color of two similarly shaped objects to reflect the color of the object that is closer to the bottom of the example. Do the same but this time make the change for the object that is closer the top. Then repeat.


Attempt 1: Incorrect
Attempt 2: Incorrect
Attempt 3: Incorrect

Participant 2

Initial description: Make corresponding shapes the same color and don't duplicate any colors.

Final description: To make the same shape the same color and not repeat any colors.


Attempt 1: Incorrect
Attempt 2: Incorrect
Attempt 3: Incorrect

Participant 3

Initial description: The size of the grid and all the patterns and shapes of blocks remain the same in the output as the input. There are no dark blue colored shapes in the output. The shapes that were dark blue in the input change colors to match colors with the other same shapes in the output.

Final description: The size of the grid and all the patterns and shapes of blocks remain the same in the output as the input. There are no dark blue colored shapes in the output. The shapes that were dark blue in the input change colors to match colors with the other same shapes in the output.


Attempt 1: Correct

Participant 4

Initial description: Same shapes, same colors.

Final description: Same colors, same shapes.


Attempt 1: Incorrect
Attempt 2: Incorrect
Attempt 3: Incorrect

Participant 5

Initial description: Copy the figure. To change the colors, recognize each figure has two of each shape. For each pair of shapes, change the color of the shape that has the leftmost color to the rightmost color.

Final description: Copy the figure. To change the colors, recognize each figure has two of each shape. For each pair of shapes, change the color of the shape that has the leftmost color to the rightmost color.


Attempt 1: Correct

Participant 6

Initial description: good

Final description: good


Attempt 1: Incorrect
Attempt 2: Incorrect
Attempt 3: Incorrect

Participant 7

Initial description: All of the dark blue is to be eliminated. The new color of the dark blue shapes will be the same color as their twin shape.

Final description: All of the dark blue is to be eliminated. The new color of the dark blue shapes will be the same color as their twin shape.


Attempt 1: Correct

Participant 8

Initial description: There are pairs of shapes that are exactly the same (no rotation). The dark blue shapes assumes the color of their partners. i.e. dark blue cross takes the color of the light pink cross.

Final description: There are pairs of shapes that are exactly the same (no rotation). The dark blue shapes assumes the color of their partners. i.e. dark blue cross takes the color of the light pink cross.


Attempt 1: Correct

Participant 9

Initial description: To transform to another color but maintain same shape.

Final description: The rule


Attempt 1: Incorrect
Attempt 2: Incorrect
Attempt 3: Incorrect

Participant 10

Initial description: Copy same as input to output

Final description: I had no idea, this is the most confusing thing I've ever attempted, and the instructions are not clear.


Attempt 1: Incorrect
Attempt 2: Incorrect
Attempt 3: Incorrect