Natasha – Mixed Reality Pterodactyl Toy

My goal was to create a mixed reality toy/playspace in which the player manipulates a physical toy to navigate a virtual environment. I was inspired to create this piece based on another project, Playtop, that I worked on earlier this year with a group of students from my Major Studio class. In Playtop, toys were set on a surface that triggered the environment to change on a screen. We created short animated loops that would play when a toy was set down. The shortcoming of this piece, I felt, was that the player could not move the toy while the video was playing. Also, the videos did not change dynamically.

For this piece, I wanted to experiment with making a toy that interacted with the environment dynamically.

Tech
I used an accelerometer and an arduino inside the toy. The toy connects to the computer via bluetooth. The environment is created in processing. I mapped the accelerometer values to the axes of a camera.

Assessment
Ultimately, this piece was not as successful as I hoped. While moving the toy does change the camera view, I was not able to get it to move through the space naturalistically. However, this was only a short project and there are several avenues I might pursue to try and improve the functionality. First, I think perhaps a gyroscope would have been better than an accelerometer for what I was trying to do. There may also be a better way to move through the space than Processing’s camera function. I am still pretty new to animating in Processing, and while this looked like the best way to change the view dynamically it may be that there are other better ways. Finally, Processing might not be the best program to animate the space. I’m not sure what programs games are typically designed in, but it seems that there might be a better option.

Processing Code

//Pterodactyl toy
//I built off the code that Joel provided for serial communication between Arduino and Processing.
//load libraries

import processing.serial.*;

//initialize port and cam
Serial port; // start the serial port

//ints for sensor data
int SensorX; // variables to hold the incoming serial data
int SensorY;
int SensorZ;
int[] x; // array of x axis values
int[] y; // array of y axis values
int[] z; // array of z axis values

PImage tree1;
PImage tree2;
PImage tree3;
PImage sky;
PImage ground;

float mapX;
float mapY;
float mapZ;

//add a font
PFont font;

void setup(){
size (800, 800, P3D);
tree1 = loadImage(“tree1.png”);
tree2 = loadImage(“tree2.png”);
tree3 = loadImage(“tree3.png”);
sky = loadImage(“sky.png”);
ground = loadImage(“moss.png”);
// FONT STUFF
font = loadFont(“Arial-BoldMT-24.vlw”);
textFont(font);
textAlign(RIGHT);

//find the correct serial port
println(Serial.list()); // print a list of available serial ports
// choose the one that is connected to the Arduino
port = new Serial(this, Serial.list()[0], 115200);// we agree to talk fast!
port.clear(); // flush buffer
port.bufferUntil(‘\n’); // set buffer full flag on receipt of carriage return

}

void draw(){
background(sky);

camera(60,mapY,mapX,
0.0, 0.0, 0.0, // centerX, centerY, centerZ
0.0, 1.0, 0.0); // upX, upY, upZ

println(mapX);

pushMatrix();
fill(0,100,0);
translate(width/2, height/2);
rotateX(radians(90));
image(ground, -3000, -3000);
popMatrix();

translate(200,0,700);
image(tree1, -1000,height/2-450);
image(tree2, 500, height/2-500);
image(tree3, 200, height/2-500);
translate(0,0,500);
image(tree1, 0,height/2-450);
image(tree2, 200, height/2-500);
image(tree3, -500, height/2-500);

translate(0,0,-500);
image(tree1, -500,height/2-450);
image(tree2, 0, height/2-500);
image(tree3, 300, height/2-500);

//pushMatrix();
//rotateY(radians(30));
image(tree1, -800,height/2-450);
image(tree2, 100, height/2-500);
image(tree3, -400, height/2-500);
//popMatrix();

//print sensor values
/*
fill(250,0,0);
text(SensorX,100,25); // draw the value of the latest data point in numbers
fill(0,250,0);
text(SensorY,100,50);
fill(0,0,250);
text(SensorZ,100,75);
*/

}

void serialEvent(Serial port){ // this function is called automatically after each draw loop
String inData = port.readStringUntil(‘\n’);
inData = trim(inData); // cut off white space (carriage return)
if (inData.charAt(0) == ‘X’){ // leading ‘X’ for X axis sensor data
inData = inData.substring(1); // cut off the leading ‘X’
SensorX = int(inData);
mapX=map(SensorX, 400, 600, 0, -1000);
}
if (inData.charAt(0) == ‘Y’){
inData = inData.substring(1);
SensorY = int(inData);
mapY=map(SensorY, 400, 600, 300, -300);
}
if (inData.charAt(0) == ‘Z’){
inData = inData.substring(1);
SensorZ = int(inData);
mapZ=map(SensorZ, 480, 530, 100, -100);
}

}

 

 Arduino Code

//I used the code provided by Joel to get Arduino to send the accelerometer values to Processing.

/*
THIS PROGRAM READS ONE SENSOR
AND SENDS THE DATA OUT AS ASCII STRING TERMINATED BY ‘\n’ (carriage return)
USE WITH DataViz512_Scale IN PROCESSING
*/
int z = A0; // Analog Sensor Reading
int y = A1;
int x = A2;
boolean sendData = true;
void setup()
{

Serial.begin(115200);
analogReference(EXTERNAL); //if using 3.5 volt accelerometer change the V range output so that the values will be in the correct range
}
void loop()
{
if(sendData == true){
z = analogRead(A0); // take a sensor reading
Serial.print(‘Z’);
Serial.println(z);

y = analogRead(A1);
Serial.print(‘Y’);
Serial.println(y);

x = analogRead(A2);
Serial.print(‘X’);
Serial.println(x);
}
delay(20);
}

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s