Sunday, 29 December 2013

Demystifying the uBlox 6 GPS Module

For an upcoming project I needed to use a uBlox GPS. Specifically, I had to use it with the Raspberry Pi, not an Arduino in Airborne mode (hmm hints about the project? :P) Even though the documentation is very informative, it still lacks a "quick start" guide for us noobs. Even though many blogs have guided me in using this module, I still had to command the GPS to work < 50,000 m (Airborne mode) and had to adjust different configs.

I'll be using a uBlox NEO-6M GPS module for this guide, but any uBlox 6 GPS would work just fine.

The unofficial uBlox 6 "Quick Start" Guide

Making the Connections

The first thing you want to do is connect the GPS to your Pi. The wiring is simple:

uBloxRaspberry Pi

Once you have made the connections, we need to enable the Pi's serial comm.

Open cmdline.txt file using nano:

sudo nano /boot/cmdline.txt

Once open and remove 'ttyAMA0', to have this:

Once you have that use Ctrl-O and Ctrl-X to save and exit.

Then edit the inittab file by:

sudo nano /etc/inittab

Once open find the "Spawn a getty on Raspberry Pi" line, and put a # before the line just after that. Restart the system.

Install GPSD by typing in:

sudo apt-get install gpsd gpsd-clients

Then start the serial port,

stty -F /dev/ttyAMA0 <baud rate>

You can use the baud rate that you want, I used 9600 for my NEO-6M

Then starts GPSD,

sudo gpsd /dev/ttyAMA0 -F /var/run/gpsd.sock

And then display it,

cgps -s

You should see an output like this:

To allow the GPS to fix and give you a location, go outside or find a place with a clear view of the sky.

Finally. kill gpsd, and let's start programming.

Using python to read/write a uBlox 6 GPS

Install two python libraries, pyserial and pynmea (I suggest you use pip, it is just easier to use that)

Once done, create a new *.py file. anywhere you want. Make sure you don't create the file as root.

Start the X GUI (startx) and open up the python file in IDLE. And type this in:

import serial
from pynmea import nmea

ser = serial.Serial('/dev/ttyAMA0',9600)
gpgga = nmea.GPGGA()
while 1:
     data = ser.readline()
     if (data.startswith('$GPGGA')):
         print 'Lat: ', gpgga.latitude
         print 'Long: ', gpgga.longitude
         print 'Alt: ', gpgga.antenna_altitude, ' ', gpgga.altitude_units
         print 'No of sats: ', gpgga.num_sats

What the above code starts a serial port, keeps on reading data until sigint and parses the NMEA strings  into Latitude, Longitude, Altitude and # of satellites.

What if you want to initialize the GPS in Airborne mode?

Create a new *.py file and open it. Type in:

import serial

ser = serial.Serial('/dev/ttyAMA0',9600)
print 'Sending set Nav airborne < 1g'
command = b'\xB5\x62\x06\x24\x24\x00\xFF\xFF\x06\x03\x00\x00\x00\x00\x10\x27\x00\x00\x05\x00\xFA\x00\xFA\x00\x64\x00\x2C\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16\xDC'

The weird hex string tells the GPS to initialize in airborne mode. Here is a list of some commands:

Revert to default config:
B5 62 06 09 0D 00 FF FF 00 00 00 00 00 00 FF FF 00 00 03 1B 9A

Enable all debug information:
B5 62 06 02 0A 00 03 00 00 00 FF FF FF FF FF 00 10 F0

Automotive Mode:
B5 62 06 1A 28 00 03 00 00 00 03 04 10 02 50 C3 00 00 18 14 05 3C 00 03 00 00 FA 00 FA 00 64 00 2C 01 00 00 00 00 00 00 00 00 00 00 00 00 6C 95

All you have to do is, replace each space with a '\x' and add a '\x' in the front and you are good to go.
These are essentially binary messages and are according to the UBX protocol.

If you want to adjust more settings, download the u-center software. Go to View>Messages View

Expand the UBX node and the CFG node. There you have it. Click on any of the child nodes, adjust the settings to want and copy the hex message in the textbox and send it over to uBlox using python.


I hope this helped someone, if I missed anything please let me know or if you have any questions feel free to ask me. Hope I saved you a few google searches. And as always,

Happy Coding!

Wednesday, 25 September 2013

Tutorial: Getting started with Computer Vision Part 2: Coding, lets put everything into practice! (Visual Basic or C#)

So in the last tutorial we understood how a computer sees an image. Now let's ask a computer to do all this stuff.

The code here is in Visual Basic, you can find the C# version at the end.


To write a computer program that would grab an image from a file that contains a yellow (maybe green!, I am color blind btw) ball, and would draw a rectangle around the ball to indicate it found it.

The Method:

Let's say we have these images:

(Right click and save them, if you are following along this tutorial)

Our aim is to draw a box around the ball. But how can we isolate the ball from the clutter. How could numbers reveal the position of the ball? Simple, we use color. The ball has a greenish-yellow color (again, I am colorblind) which can be indicated by the color of the pixels which are yellow greenish. In RGB this is around, 134,106,32, which looks like this:

So lets start! If you are not using my images, then make sure you have found out the average color of the ball/object you are trying to use, else, stick with the values I found out.

Go ahead, start with a new project and add a picture box to it (assuming you named it pbDisplay). This picturebox will serve as a display for the images.

In the Form_Load event type (yeah type, don't copy!) out the following:

Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        Dim bmImage As New Bitmap(Filename Goes Here!!)

        pbDisplay.Image = bmImage
End Sub

The bmImage as a "Bitmap" object. It is essentially a group of pixels, an image. The constructor of the bitmap object takes in a string, which refers to the file path of the image you want to load to the memory.

Once we have our bitmap, we simply show it in the picturebox.

Add the path to your filename and launch your program, you should have an image loaded onto your form. Congrats!

The Method: The real thing :D

Well now the easy part is done, we'll now try to detect the ball.

The basic algorithm is to iterate every pixel and determine weather that pixel is green or not.

To determine if a specific RGB values is green or not, we shall write a function as follows:

Dim BALL As Color = Color.FromArgb(134, 106, 32)
Const THRESH As Integer = 15
Function isPixelGreen(ByVal pixel As Color) As Boolean
        If (pixel.R <= BALL.R + THRESH And pixel.R >= BALL.R - THRESH) And (pixel.G <= BALL.G + THRESH And pixel.G >= BALL.G - THRESH) And (pixel.B <= BALL.B + THRESH And pixel.B >= BALL.B - THRESH) Then
            Return True
            Return False
        End If
End Function

BALL is a variable that is the average color of the ball. (Remember a color has the RGB components). As the actual pixels of the image of the ball will not be exactly the average, we keep a THRESH, which acts like a range.

If the color provided (pixel) is in the range, we return true else false.

Now all we need to do is iterate through every pixel and check if the pixel belongs to the ball or not. We can right a simple nested for loop under the declaration of the bmImage object:

 For x = 0 To bmImage.Width - 1
            For y = 0 To bmImage.Height - 1
                If isPixelGreen(bmImage.GetPixel(x, y)) Then
                    bmImage.SetPixel(x, y, Color.White)
                    bmImage.SetPixel(x, y, Color.Black)
                End If

Here we iterate through all the x's and y's of the image and using the GetPixel method, we were able to extract the pixel color from the image, pass it through our function. If the function returned true, we color it white else black (using the SetPixel method).

Run the program and you should see a black and white image. Woah! The image wasn't right? No problem, try increasing the threshold to about 25. This will make the checking less biased. Is the image better? It should look something like this:

So we see the ball is white but there is noise that is white too! For a beginner tutorial this is fine, as we progress, we will be able to do this almost perfectly using advanced algorithms, but for now this is okay.

Drawing the Box!

Now we need to define a rectangle around the ball. For that we need a start position and an end position.

We will fetch that by modifying our for loop:

        Dim startPos As Point
        Dim endPos As Point
        For x = 0 To bmImage.Width - 1
            For y = 0 To bmImage.Height - 1
                If isPixelGreen(bmImage.GetPixel(x, y)) Then
                    If IsNothing(startPos) Then
                        startPos = New Point(x, y)
                        endPos = New Point(x, y)
                    End If
                End If
        Dim rect As New Rectangle(startPos, New Size(endPos.X - startPos.X, endPos.Y - startPos.Y))

Here we define our start and end positions and set the start position to the first green pixel and the end as the last green pixel.

Then we define a rectangle using the start and end positions. Finally we need to draw a rectangle and this can be done by:

        Using g As Graphics = Graphics.FromImage(bmImage)
            g.DrawRectangle(Pens.Blue, rect)
        End Using

Click launch and play with the threshold and color, you should see something like this (threshold: 23)

Well, this isn't impressive: but its a start. It is indeed difficult for a computer to make conclusions out of images. (If you try the other images, the noise is so much that the rectangle is way off!!)

But don't be sad. I guess we learnt a lot and as we progress through this series, we will use more advanced algorithms to get amazing results.

Stay tuned for the next part: improving the results using blob detection.

Till then,
Happy Coding

Oh, here is the C# version :D

using Microsoft.VisualBasic;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Diagnostics;
public class Form1

 private void Form1_Load(object sender, EventArgs e)
  Bitmap bmImage = new Bitmap("<filename>");

  bool firstPass = true;
  Point startPos = default(Point);
  Point endPos = default(Point);
  for (x = 0; x <= bmImage.Width - 1; x++) {
   for (y = 0; y <= bmImage.Height - 1; y++) {
    if (isPixelGreen(bmImage.GetPixel(x, y))) {
     if (firstPass) {
      firstPass = false;
      startPos = new Point(x, y);
     } else {
      endPos = new Point(x, y);
  Rectangle rect = new Rectangle(startPos, new Size(endPos.X - startPos.X, endPos.Y - startPos.Y));

  using (Graphics g = Graphics.FromImage(bmImage)) {
   g.DrawRectangle(Pens.Blue, rect);

  pbDisplay.Image = bmImage;
 Color BALL = Color.FromArgb(134, 106, 32);
 const int THRESH = 23;
 public bool isPixelGreen(Color pixel)
  if ((pixel.R <= BALL.R + THRESH & pixel.R >= BALL.R - THRESH) & (pixel.G <= BALL.G + THRESH & pixel.G >= BALL.G - THRESH) & (pixel.B <= BALL.B + THRESH & pixel.B >= BALL.B - THRESH)) {
   return true;
  } else {
   return false;
 public Form1()
  Load += Form1_Load;

Tutorial: Getting started with Computer Vision Part 1: The basics, how could a computer see.

/!\ Warning: The following is meant for a person who knows programming and wants to start with computer vision without reading through the intimidating things!

Wouldn't it be cool if computers could actually see like humans? I mean to a human this question seems absurd but we must understand the this trivial task of seeing isn't really trivial for a computer.

Let us look at an example.

The above is a bad sketch of the eye. :D, but it pretty much sums up how we look. Its simple, light enters the cornea (the whitish part) into the pupil and then to the retina. The retina is where the magic happens. As the light hits an individual receptor on the retina, it sparks off an electrochemical reaction. This information about light gets collected and transmitted to the brain for further processing.

Now lets see how a computer would get this data.

This is a bad sketch (again!) of a webcam. Light enters through a series of lenses which hits the shutter. Then it hits the sensor, which does the same job as the retina. But unlike the retina, its not organic, its made up of silicon. :D. The shutter just acts like a gate, if it opens, light hits the sensor, else it doesn't.

So so so so, what does this magical sensor give us in terms of data? Numbers. Yeah. Numbers. Think of an image like a group of pixels. Each pixel has its separate color

Essentially, the pixels that you can see on the zoomed in version is what makes an image.

Well, what makes a pixel? How do you devise a system that could produce any color in the visible spectrum? What are the components of a color? Well, the answer is: most of the colors can be produced by mixing the colors, Red, Green and Blue. (RGB). Each color element of RGB occupies 1 byte of memory. i.e. 256 different values. So a (R=255, B=0,G=0) would produce a red color. Click Here to try making your own colors using RGB values.

So let's answer the question we asked earlier, how could a computer see? Well, to a computer an image looks something like this:


And these are just 12 pixels! An actual image has thousands of pixels.

Well, that is the end of this tutorial, if you wanna start coding, check out Part 2!

Friday, 29 March 2013


The new algorithm works (still a secret) Here is an application screenshot, showing the robots location and pose.

I shall be uploading the source code once I have coded it properly, and will be making a "Kinect Robotics" library for anyone who wishes to use the Kinect for robotics using the "official" SDK.

Wednesday, 20 March 2013


After days of struggle, I think I have struck oil. The algorithm I mentioned works like a charm, but still needs to be tested. I shall be posting the code soon for anyone like me (Lol, I hardly think it'll happen). Anyways, I will be creating a Kinect Vision Library, which will allow .NET devs to easily access whatever I have without  the struggle....

Here is the function I use, any questions, please comment below.
Code: VB

Function estimateDistBWCentroid(ByVal srcIn As PointCloud, ByVal dstIn As PointCloud) As Double
        Dim src As PointCloud = srcIn
        Dim dst As PointCloud = dstIn
        If < 3 Then
            Throw New Exception("Error: at least 3 correspondences must be provided")
        End If

        Dim cL, cR As New Point3D(0, 0, 0)
        Dim S, N As MatrixA
        Dim Z, D As MatrixA

        Dim v As New List(Of Double)

        Dim nMatches As Integer =
        'Compute centroid
        For i = 0 To nMatches - 1


        Dim F As Double = 1.0 / nMatches

        cL *= F
        cR *= F
'This will give the transitional vector (just change z to x or y to get the corresponding vector)

        Return cL.z - cR.z 'System.Math.Sqrt(((cL.x - cR.x) ^ 2) + ((cL.y - cR.y) ^ 2) + ((cL.z - cR.z) ^ 2)) '
End Function

Tuesday, 19 March 2013

New Ideas

Its been days since I am trying to find an optimal algorithm to find the transformation between two point clouds with outliers and I have failed so far. I have a new idea.

  • Compute the centroid of the two point clouds (least means squares)
  • Find the displacement of the centroid
  • Repeat these with "hypothetical inliers" and revise the error (RANSAC)
This would give me a pretty good approximation....

Programming in a better environment

So, I have my holidays, and yeah, I must complete Yoda (its a friendly robot) in this time but I am not doing this at my house, I happen to visit my Grandparents at Dehradun and this is the last day I am here, the climate here is absolutely amazing. So much greenery opens one's mind. Here are a few pictures.... (from my iPod)