EyeToy Vision – facial recognition using the PlayStation 2 EyeToy camera

The PlayStation 2 EyeToy was released in 2003 and was basically a USB webcam that you could attach to your PS2 to play certain games using your body and voice commands. For my current job, I have been developing a strength training application using Microsoft’s Kinect for Xbox One which piqued my interest in computer vision. I started messing around with facial recognition on the Kinect using HD face as well as a more traditional PCA based approach using Eigen Faces from this great sample here. I then remembered that I had an old EyeToy USB camera laying around at home that I could use as a capture device and use OpenCV for some basic face recognition. I started work on it last night and now have an working(ish) example of how to use the EyeToy for facial recognition using OpenCV. Bear in mind this is the first time I have done a VC++ project so I probably didn’t do everything in an optimal way.

screenshot

Requirements:

-EyeToy USB Camera (The one I used was an early model and was manufactured by Logitech, later EyeToys were manufactured by Namtai. I have not tested the driver with a Namtai made EyeToy but I am pretty sure it will work.)

-An open USB port (does not work properly with USB hubs)

-Visual Studio 2015 or higher (I compiled with VS 2017 Enterprise)

-EyeToy Vision Source Code: https://shravanj.com/files/EyeToyVision.zip

Before we can begin discussing the program, we need to install the EyeToy drivers, since there is no official PS2 EyeToy driver for Windows. I followed this guide and got it working on Windows 10: http://metricrat.co.uk/ps2-eyetoy-on-windows-8-64-bit-working/

You’ll need to download the driver (which I have uploaded to my website for your convenience here) and extract it. Once you have done that open up a command prompt window with administrator permissions. We need to temporarily disable driver signing verification so we can install the driver. Enter in the following commands:

bcdedit -set loadoptions DISABLE_INTEGRITY_CHECKS

bcdedit -set TESTSIGNING ON

From the Device Manager, find the EyeToy, right click and select Update Driver. Select let me pick from list -> have disk -> locate the unzipped driver folder and select HLCLASSIC.inf. Click continue when prompted about the unsigned driver. Once finished, re-enable driver signing enforcement like so:

bcdedit -set loadoptions ENABLE_INTEGRITY_CHECKS

bcdedit -set TESTSIGNING OFF

Verify the driver works by opening up the testing application inside the driver folder.

Now that we have the driver setup and ready, we need to prepare Visual Studio. In my initial stages of development, I tried linking OpenCV directly to VS but I never got it to work properly, so instead I found a NuGet package that manages the whole thing for me. Named opencvcontrib, it contains a x64 compiled version of OpenCV 3.1 and more importantly includes the contributed modules which contains the FaceRecognizer class which is not found in the stand alone version of OpenCV. In order for this to work, we need the Visual Studio 2015 platform tools because that was what the whole OpenCV source was built against. If you are using VS 2015 you do not need to do anything, but if you are using VS 2017 like me, you will need to install the 2015 platform tools. To do this, go to Start Menu > Visual Studio Installer. Click the menu icon for you installed VS 2017 product and select Modify. Open the Individual Components tab and scroll down to Compilers, build tools, and runtimes. Select the “VC++ 2015.3 v140 toolset (x86,x64)” and install it. You are now ready to compile the program.

I will get more in depth on the actual programming in another post, I just wanted to share my initial progress on this project.

 

 

Advertisements

Recommended Software: Kaspersky Anti-Ransomware for Business

I generally don’t recommend using anti-virus apart from Windows Defender and Malwarebytes Anti-Malware, but due to recent developments it seems like protection against ransomware is needed. I am aware that a future Windows 10 update will allow protection against ransomware in the idea that you can protect specific folders from unauthorized programs accessing it. This, along with Windows Defender Application Guard seem to be a good combination to thwart ransomware. Of course, the best technique is simply following general internet security rules: never open or download files from any unknown email senders or sketchy websites. Use websites like Virus Total when opening an potentially risky file. But if you want an extra line of defense, look no further than Kaspersky Anti-Ransomware. This free download (yes, it is free, unlike most Kaspersky security products) provides solid ransomware and general malware protection via real-time scanning (something that the free version of Malwarebytes does not provide). In essence, its basically a stripped down and free version of Kaspersky Internet Security, a product I used to use a few years ago but never really deemed necessary considering I needed to purchase a new subscription every year. In today’s state of computer security, you can never be too careful. You can find a download link to Kaspersky Anti-Ransomware for Business below (you don’t actually have to own a business to use this software) along with a review from AvGurus posted on the MalwareTips forum.

Link for download: https://go.kaspersky.com/Anti-ransomware-tool.html

Link for review: https://malwaretips.com/threads/kaspersky-anti-ransomware-tool-for-business-quick-test.62451/

 

Why your Xamarin Forms 2.3.x project stopped working in Visual Studio for Mac

(Originally published 5/11/17)

On May 10th, 2017 Microsoft launched the public release of Visual Studio 2017 for Mac. It is intended to be a replacement for Xamarin Studio as it integrates .NET Core along with Xamarin and cross platform game development components. From a UI standpoint it looks very similar to the latest version of Xamarin Studio, but under the hood a few key changes are lurking. The one I am concerned about is the switch from xbuild to MSBuild as the build system for Xamarin.iOS. MSBuild was optional before but is now the default system. For many projects, the transition is seamless, but for those who are using the outdated 2.3.x versions of Xamarin.Forms, a huge problem is evident: it does not support MSBuild at all. In fact, many builds of 2.4.x do not work, but the latest version guarantees compatibility. This was a rather perplexing problem that took me a while to figure out because I had no idea why the build process was failing in both Xamarin Studio and Visual Studio for Mac. Since the VS for Mac installer bundles the latest versions of Xamarin.iOS and Mono MSBuild is the default in both VS and Xamarin if you use them side by side on the same machine. Trying to build a project that uses the 2.3.x version of Xamarin.Forms will fail with a xbuild failure to start error on VS and a missing pdb file error in Xamarin Studio. This could be avoided if Microsoft warned about potential incompatibility with apps that rely on xbuild. This leaves me with two options: roll back to an earlier version of Mono and Xamarin.iOS (which is basically uninstalling everything and reinstalling Xamarin Studio which is a huge pain), or figuring out how to safely update to the latest version of Forms. This is tricky since there are many crucial packages in my solution that rely on this specific version of Forms, so all of them will need to be updated without breaking functionality or legacy code. This is the only way to ensure things will work properly in the future since Microsoft has stated that VS for Mac will be the future for Xamarin. This means that in order to stay up to date with Xamarin, we need to be using the latest version of Forms. This will not be an easy task, but who said cross-platform development was easy?

Update macOS through Terminal

(Originally published 4/26/17)

Downloading updates in the Mac App Store, especially Xcode, has always given me problems. Recently I discovered that you can actually update system software and 3rd party apps via Terminal using the “softwareupdate” command. To list the updates, issue “softwareupdate -l” and install them with “softwareupdate -iva” which will install all available updates. If you want to install a specific update, just use “softwareupdate -i ‘name of update'”. Make sure you have sufficient user privelages before installing.

Xamarin Tips and Tricks – Custom push notification sounds on iOS and Android

(Originally published 4/21/17)

Upon the receiving a push notification, iOS and Android will automatically use the default tone for the notification (or just vibrate if your device is set to silent). Normally, you would have to change the notification tone within settings, but if you want your app’s notification tone to stand out from the default tones, you can set your own notification tone. Today we will look at how to implement this in Xamarin.Forms (you will need a dependency service for this to work since the Android and iOS solutions use platform specific code).

iOS

Before you can use your custom notification tone, you need to have it in the .caf format for it to work inside of a push notification (MAKE SURE YOUR SOURCE FILE IS 30 SECONDS OR LESS IN DURATION). To do this on macOS, issue the following command in Terminal: afconvert -f caff -d LEI16@22050 input.mp3 output.caf. Now that you have converted the file, you need to add it to your iOS Resources directory. Inside Xamarin Studio, under your iOS solution right click on the Resources folder and click “Add files” and select your converted .caf file. Now that you have the file converted, we can build a local notification with the custom tone:

IMPORTANT NOTE: iOS limits 3rd party apps on how long notification sounds can play to 30 seconds. Banner style alerts (the default style) only allows a few seconds, so the user will need to change the alert type to “Alerts” under Settings > Your App > Notifications > Alert Style When Unlocked. You will also need to make sure your sound file itself is only equal to or less than 30 seconds in duration.

Android

You can use a .mp3 file of any duration you see fit because we won’t actually be using the notification’s tone payload to play the sound due to some quirkiness in the way Xamarin handles custom notification tones on Android. We will actually build a notification and then use Xamarin MediaPlayer to play the tone when the notification fires. First we need to create a new folder inside of the Android solution’s Resources folder. Right click on the Resource folder and click “New folder”. Name the folder “Raw”. Now add the .mp3 file into the Raw folder. The gist below shows basic notification setup:

IMPORTANT NOTE: Before compiling, do a clean and rebuild of the project using right click -> “Clean” and right click -> “Rebuild”. This will solve issues involving the resources folder not being compiled properly. Highly recommended every time you add new files to the folder.

Xamarin Tips and Tricks – Clearing Lists in C#

(Originally published 4/3/17)

Sometimes, using the built-in Clear() function on C# Lists may not always work. For example, I needed to clear a list of all its contents every time I returned to a Page view. Using the Clear() function was not clearing the list properly and thus resulting in the Page crashing. Although it may not always be the most obvious solution, if you are every getting IndexOutOfBounds exceptions when dealing with clearing a list, you may want to try doing it the old fashioned way – using a for-loop to traverse the List from head to tail and using the RemoveAt function to remove the object. Here is a snippet of the function below:

This simple function can solve some major issues regarding clearing of a list. This can really be applied to any language in any situation where the built in clear function is not doing the proper job. Thanks for reading, and stay tuned for more updates.

Xamarin Tips and Tricks – Compressing Videos in Xamarin.Forms Projects

(Originally published 3/29/17)

Today I will be starting a series of Xamarin programming tips and tricks that I have learned from different projects I have worked on. Lets kick this off with platform specific video compression in Xamarin.Forms. Since media encoding is done differently on iOS and Android, we need to write different solutions for Forms projects. We will be taking a look at iOS media compression today. Our media will be picked using Xamarin’s Media Plugin (Plugin.Media), which delivers the picked media in a MediaFile object. We will then pass this object into our compression method which will then send it back out as a file stream. You can view a gist of the method involved below:

Your MediaFile will have its uncompressed mov file taken out of it, encoded again with higher compression at a lower resolution using the native AVAssetExportSession, and sent back out as a stream. All of this runs in async and is fast on new(ish) iOS devices. You can change the amount of compression applied by modifying the export preset as well as change what type of object the file gets sent back out as (a stream is the easiest to work with especially if you are planning on uploading the file to a web service). These methods can be easily dropped into a platform specific class using a dependency service to call upon it when needed. Also note that you will have to create a public interface for this as well.

 

Fixing Raspbian apt source list

(Originally published 3/28/17)

Updating via apt suddenly stopped working for me today on both my Raspberry Pis with one on Jessie and the other one on Wheezy (it got stuck on waiting for headers). Took me a while before I figured the mirror director was the problem (I guess there is a outage with one of the mirrors). Inside the apt source list, commenting out the mirror director entry and replacing it which archive.raspbian.org fixed it.