Tag Archives: Documentation

Why blogging can help organise your mind and open up unexplored avenues

This is yet another blog that has changed since I started writing it.  It was to be “Unity 5 and my failed attempt to access facial tracking using OpenCVSharp.”  I will now try to show why blogging can help to organise your mind and leave you open to possible solutions.  I will still tag this blog with Unity 5, OpenCVSharp, Webcam and facial tracking, just in case someone else is having problems with getting it to work in Unity 5.  The first part will briefly explain how I got to the stage before I got facial tracking to work in Unity and then explain how this process helped me find the solution.

Firstly, if you have a webcam attached to your computer and aren’t sure if it is working, this Windows trick will save you a heap of time.  Go to your start bar and type in “Camera”.  You can then click on the Camera button.  If it asks you to connect a camera, return to the main screen and press Fn-F6.  This will turn your camera back on and you should be right to go.  Now if you click on the camera, your ugly mug should now come up on the screen and you can take a quick photo of your self satisfied look after saving some time on Google.  Please do it, because that will be the last time you will see that look for some time.

Let me explain why this is the case.  Unity 5 is a 64 bit program and will need 64 bit versions of all the relevant DLLs. More commonly, these are 32 bit and they will not work in Unity 5. But the 64 bit versions can be found at this link: https://github.com/shimat/opencvsharp/releases

My search for trying to find a working version of Facial tracking led me to this site: http://forum.unity3d.com/threads/opencvsharp-for-unity.278033/ and from there I was able to download the relevant Unity package.  It was obvious that it did not include facial tracking and without more research, I wouldn’t get it to work.

I probably spent more time than it deserved, but finally found a working script that tracked faces from this obscure site: http://ux.getuploader.com/aimino/edit/3

It worked perfectly in Unity 4.x, but directly importing it to Unity 5 gave me errors.  I figured that this would be due to the 32 bit DLLs that the original project held.  I changed the “OpenCvSharp” and “OpenCvSharp.MachineLearning” DLLs to the 64 bit version, but ended up with an error, something along the lines that there was an error reading from the “haarcascade_frontalface_alt.xml” file.

I gave up at this point, because there was so much other work expected from me, and this was a prototype for a game that was using a pulse monitor that seemed to be working quite well.  My lecturer convinced me that I should do a blog about this failure so that it would 1. show the work I had done on researching this problem and 2. give anyone else the option to see what I had done and perhaps save them some work on researching and getting to the same point.

Because some time has gone by, I really can’t remember exactly what steps I went through to get to this point, but the ones that I do remember are listed above.

As I was trying to recreate this, I was trying to work out what I had and hadn’t done, so I made a copy of the Unity 4.x version and re-imported it to Unity 5. I then replaced the DLLs with the 64 bit version (as mentioned above).  Then I wondered if there were any other sneaky DLLs that might need replacing. I did find some more DLLs. There were five DLLs that I tried to find 64 bit versions of but they were only available in 32 bit and the fact sheets seemed to explain that they were good to use in a 64bit situation.  Then I found “OpenCvSharpExtern.dll” which I know can be a 64 bit dll, but was still a 32 bit.  I replaced it with the 64 bit version and tried to run the face tracking again.  Face tracking is now operational in Unity 5.

Face detection working in Unity 5

Face detection working in Unity 5

There is a slight bug, however.  It will work the first time you play a scene, but stopping it and trying to run it again will cause Unity to freeze, or become unresponsive.  This is likely to the one red error that it produces at the start of the run.

The error itself reads:

MissingFieldException: Field ‘OpenCvSharp.DisposableCvObject._ptr’ not found.
FaceDetectScript.Start () (at Assets/Scripts/FaceDetectScript.cs:43)

The error relates to this line of code found in the FaceDetect script:

CvSVM svm = new CvSVM ();

The main things I found trying to research this error are https://github.com/shimat/opencvsharp/blob/master/src/OpenCvSharp.CPlusPlus/modules/core/Ptr.cs and https://github.com/shimat/opencvsharp/blob/master/src/OpenCvSharp/Src/DisposableCvObject.cs .

They both appear to be part of shimat’s source code for creating the DLLs.  I am horrible with pointers in C++ and didn’t think that they were supported in C#, so I will try to find out a solution to this problem and will edit this post within 2 weeks.  NOTE: It will be below this with some sort of edited heading.

*****EDITED 21/07/15*****

The solution seems to be to toggle the following lines of code:

CvSVM svm = new CvSVM ();
CvTermCriteria criteria = new CvTermCriteria (CriteriaType.Epsilon, 1000, double.Epsilon);
CvSVMParams param = new CvSVMParams (CvSVM.C_SVC, CvSVM.RBF, 10.0, 8.0, 1.0, 10.0, 0.5, 0.1, null, criteria);

They are not being used elsewhere in the program and seem to be somewhat incomplete in their use.   CvSVM is a Support Vector Machine which is given labeled training data.  An algorithm outputs an optimal hyperplane which can categorise any new examples.

CvTermCriteria means the termination criteria for the iterative algorithm that the CvSVM is using and the CvSVMParams are the parameters used to train the CvSVM.

Usually there is a “Train” and “Predict” method called to the SVM to get it to function and adequately predict something.

I have no idea why this code was left in the project but it is obviously incomplete and the facial tracking works without them being included.

**********

This project had frustrated me in several ways.  Firstly, that for so long, I thought i was so close to having it working.  Secondly, I spent too long on it, without making accurate notes and recording my results which makes it harder to prove what you have done for research into the subject and third, if I had done the above, I could have stumbled on the solution a lot sooner.

What I aim to do in the future, and I highly recommend to others, especially any students, is to record what is happening with your research, even if you save it as a draft.  Actively copy links and what you are looking at, or for, as you go.  In this way, it is all laid out and you might be able to see holes in your logic or thinking as you progress.  At worst, leave it for a week or two and approach it with fresh eyes.

When communication and GDD’s aren’t enough

For weeks, I have been designing assets and scripts believing that my team’s current project,  “Atl and Ollin”, was to be one single open world.  Two nights before the first play test, I find out that it will be 3 separate levels.  Apparently, it has been so for some time, but I wasn’t aware of it and it isn’t covered in the GDD.

I don’t know if I made an assumption about the state of this game, but I believed that my concept was valuable to the desired outcome of the game which is the need to build trust.  There would have been no breaks from the immersion of the characters and their journey across the landscape.  I knew that there were 3 different stages of the game, but I don’t believe that they should be broken up with different levels.

I will endeavour to try and convince the designer to consider this idea and try to compare it with life itself, where there is a stage of learning and playful abandon (tutorial area), followed by the panic of the middle years (disturbing the fog) followed by the inevitability of death (final area).

I didn’t even have a chance to even write up this blog as the pedal was to the metal all weekend.  Sleep deprivation..who needs it 😉

How my TDD and Repositories have evolved between the 1st and 2nd group projects.

Link to my WWW TDD:

https://docs.google.com/document/d/1qwcX78ve2fH4vHgdw31LDY-Td6p74gws107fcStWWBc/edit?usp=sharing

My first attempt at creating a repository went very well.  I was using Bitbucket with SourceTree and created the repo easily.  The problem came when I first tried to push the initial project to the repo.  I had what seemed to be a catch22 error.  Couldn’t push because I hadn’t pulled, and couldn’t pull because I hadn’t pushed … at least that is how I remember it.  Corey was my programming partner for this project and we quickly changed over to GitHub for windows.  This went OK up until the last day when the repo crashed and nothing could be done.  Pat Monaghan quickly amended our “gitignore” file and fixed it, just like that.  *sigh.

Our TDD reflected my use of Bitbucket, but did not evolve when our need to change repositories occurred.

This was not a “living” document …. and on reflection, neither is my second one, but we will get to that shortly.

The document itself, is full of useless crap from the template, that should have been deleted.  There are no risks documented and considering the genre of this project, there should have been some risks.  Player not understanding the puzzles, player not understanding the way to transition between characters.   These were obvious after play testing and should have been documented then, at least.

The reason why this, and the next TDD wasn’t a living document, is because it was not a priority.  To be totally honest, the first time I looked at these documents, since creating them, was just now, before writing this blog.  That makes me a little sad, because the work and thought that went into them was wasted.

 

Link to my Valour TDD:

https://docs.google.com/document/d/1qwcX78ve2fH4vHgdw31LDY-Td6p74gws107fcStWWBc/edit?usp=sharing

The thought and effort in my second TDD is more apparent.  The only useless thing from the template at the start of the audio section.  The remainder is concise, to the point and relevant to this project.

But again, it is not a living document because the concept about parallax scrolling of the scene and Lists/arrays for checkpoints throughout the level are still mentioned.

It is interesting to note that I did identify three risks and that each one of them came to fruition.  I naively did not identify the risk that team members would let the project down by leaving the art assets until the very last moment.  In hindsight, it could have been foreseen.

But this risk section needs to do more than merely identify the risks.  It also needs to identify the ways that the risks can be averted.  References to the GDD document would be applicable for the three risks I identified, and if there was nothing in the GDD, then it needs to be amended to make sure that my risks can’t be traps for the team to fall into.

As to the risk of team members letting the team down.  I believe that there should be a detailed, documented course of action for keeping the team on track and dealing with members that are not pulling their weight.

As to making it a living document, I believe that any changes can be made through using the strikeout feature and re-writing sections, at least for the duration of this course, so Lecturers can see that the document has changed over the length of the project.  As the document’s creator, it is my responsibility to make sure that any changes affecting the document should be discussed with all the other members of the group and bringing in the Game Producer if there is any lasting conflict over the changes.

Most of the content needs only be done the once and then reproduced for future projects, with minimal changes.  For example, a tutorial for using github only needs to be done once and only changed when a flaw has been identified, which only means that you have a new version for the project after that.  The same can be said about the process for checking if team members are pulling their weight.

My second attempt at creating a repository was a lot better. There were no conflicts with pushing, pulling or merging.  I don’t know if that it because I am still using the essence of Pat’s gitignore or whether everyone was more aware of using a repo.

Two think I will take out of this, is that there need to be more documentation about using the repo in the TDD.  There needs to be more than just supplying a link to it.  There should be provision for what to put in the comments when pushing.  No comments, or “The name explains it”, are essentially useless.  Full details should be provided.  The documentation should take a new comer by the hand and help them create, load, clone and use the repository.

While there has been a marked improvement between the two documents, there is a lot more room for improvement.