One of my tasks this Trimester is to get feedback and use it, from something other than standard game controllers or the mouse.
I have teamed up with designer, Savik Fraguela (yet again), and I am providing bio-feedback for his game. In order to keep it inexpensive at this stage, I am using feedback obtained from an Arduino board and software that comes with the “Pulse Sensor” from http://www.pulsesensor.com.
This meant that I needed to download and install several Software suites and the FDTI drivers. The drivers were a pain in my case because I am using Windows 8 and this operating system hates using unverified, or unsigned drivers. I had to get past that by following the directions on this site : https://learn.sparkfun.com/tutorials/disabling-driver-signature-on-windows-8
Installing the rest of the software was uneventful as was attaching the monitor to the Arduino board.
I was able to get the software (called a sketch in Arduino Studio) running and pushed onto the board. I was also able to get the serial driver showing what feedback was comint through the port.
My first problem came when trying to write my Unity script. I eventually had errors saying that the serial was not open. By this stage, I had to return the Arduino hardware to my Lecturer, but I suspected that it was because ArduinoStudio was using the port to read the information.
This was confirmed over the weekend. I again ran Arduino Studio but didn’t display the Serial Port in that suite. Unity had no problems with the set up.
Now the fun part. I am used to manipulating gameobjects, and not so much strings and byte and char.
I was trying to put the data from the serial port into a char array and specifying the length as 5, because the data being sent was a Letter that indicated what sort of information was being sent, then an int of 2 or 3 characters and then a carriage return. I was able to display only the numbers and nothing else. The code I was using was serialPort.ReadByte().
I eventually found (thanks Google) that I should be reading using serialPort.ReadLine() and copying that to a string. This allowed me to read the whole line of data coming over the serial port in the format of S203, etc.
The incoming data was split into three categories. Data that starts with an S prefix which gives the user raw data. Data with a Q prefix that is apparently some sort of time measurement from the last beat and data that has a B prefix, which is the one I wanted as it is the beats per minute. This is taken over 10 beats and averaged out for a minute.
c# strings are so easy to work with. I thought that I would have to split the string down into a char array and try to work with that, but as it turns out, a string is already a char array. Each element can be dealt with as string, string, etc. That was fine until I tried to use this to find out the start of the string .. if(string == “B”). I was told that Unity couldn’t compare a string with a char.
More searching and I found that you could do this if(string.StartsWith(“B”)). This has me beat. It sounds the same as if(string == “B”), but Unity accepted that and it worked. I was now getting all of the data that had the B prefix. I did notice that the information seemed to be fluctuating pretty steadily, and occasionally wildly.
Because there was no guarantee as to how long the ints were, they could be 2, 3, or 4 numbers long(although in the case of the B prefix, it is fair to assume that they will be only 2 or 3 numbers long) I made use of string.Length, they are arrays after all.
As only the B data is coming through to this point, I then created a new temp string and appended it with the contents of my original string, excluding the prefix. I then converted that temp string into an int.
In order to try and settle down the fluctuations, I created a List of ints and when the count reached 5, it would take the average of those bpm and return this new bpm. (after removing the first member of the List)
I know that Savik will be using an enum to gauge the factor of how scared the player is and using that enum to ramp up the fear inducing elements of his game. If this list isn’t stable enough, I will look at making the list longer and even consider taking the highest and lowest values out, then averaging the rest and using that information for the new bpm.
In conjunction with this, we were considering using OpenCV and OpenCVSharp (and even some other formats) to use facial tracking from a web cam and supplement information from them to aid in the bpm. I will write another blog about this regardless of whether I can get it working in Unity 5, so that someone can see the effort I have put into trying to get it working and perhaps show me if and where I went wrong. I did find a project that will work in Unity 4, but there seems to be a problem with Unity 5 reading the haars_Cascade_facial .xml file.
In the mean time, here is the file that I used to get the pulse monitor working with in Unity:
public class SerialListener : MonoBehaviour
public SerialPort serial = new SerialPort(“COM3”, 115200);//create new serial port
private string beat = “”;// string to hols the data in
private List<int> beatPM = new List<int>(); //List to hold the beats
void Start ()
//open a new connection
void Update ()
beat = “”;
//Read data input … should be a letter to start (char)
//then 3 numbers and then carriage return
beat = serial.ReadLine();
// print (beat);
//Create an int that can be used by the Client
int somethingToSend = AnalyseBeats(beat);
//Function connecting to Arduino
public void OpenConnection()
if (serial != null)
print(“Closing port, because it was already open!”);
//message = “Closing port, because it was already open!”;
serial.Open(); // opens the connection
serial.ReadTimeout = 1000; // sets the timeout value before reporting error
// message = “Port Opened!”;
print(“Port is already open”);
print(“Port == null”);
int AnalyseBeats(string str)
//Data comes in with a “S” header, a “Q” header or a “B” header
//S = raw data .. “Q” is (allegedly) the time since the last beat was detected
//and B is the one that we want. It is the beats per minute averaged over 10 beats
// print (“Original data.” + str);
//create a new string
string t = null;
//append to t everything except the starting letter
for(int j = 1; j<str.Length; ++j)
t += str[j];
//Convert that string into an int
int temp = int.Parse(t);
// print (“Modified data.” + temp);
//add that int to the list
// print (“BMP.count.” + beatPM.Count);
//Create an average of the BPM to give a more stable reading of the current pulse rate
int bAv = 0;
for (int i = 0; i < beatPM.Count; ++i)
bAv += beatPM[i];
bAv= (int) bAv/beatPM.Count;
// Debug.Log(“Average over last 5 beats = ” + bAv);