OpenCV Python script from Gimp procedure - Grass/Hard surface edge detection

I would like to develop a Python OpenCV script to duplicate/improve on a Gimp procedure I have developed. The goal of the procedure is to provide an x,y point array that follows the dividing line between grass and hard surfaces. This array will allow me to finish my 500 lb 54" wide pressure washing robot, which has a Raspberry Pi Zero (and camera), so that it can follow that edge at a speed of a couple inches per second. I will be monitoring and/or controlling the bot via its wifi video stream and an iPhone app while I watch TV on my couch.

Here is a sample original image (60x80 pixels):

The Gimp procedure is:

1. Convert image to indexed 2 colors. Basically grass on one side and bricks or pavement on the other side. DARN SHADOWS oops that's me :)

2. Of the two colors, take the lower Hue value and magic wand on a pixel of that value with the below wand settings. The Hue setting of 23 is how I remove shadows and the feather setting of 15 is how I remove islands/jaggies (grass in the cracks :).

3. Do an advanced selection to path with the following advanced settings values (changes from default values are yellow). Basically I want just line segments and my (x,y) point array will be the Yellow path dots.

4. Next I export the path to an .xml file from which I can parse and isolate the yellow dots in the above image. Here is the .xml file:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>  <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN"                "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">    <svg xmlns="http://www.w3.org/2000/svg"       width="0.833333in" height="1.11111in"       viewBox="0 0 60 80">    <path id="Selection"          fill="none" stroke="black" stroke-width="1"          d="M 60.00,0.00             C 60.00,0.00 60.00,80.00 60.00,80.00               60.00,80.00 29.04,80.00 29.04,80.00               29.04,80.00 29.04,73.00 29.04,73.00               29.04,73.00 30.00,61.00 30.00,61.00               30.00,61.00 30.00,41.00 30.00,41.00               30.00,41.00 29.00,30.85 29.00,30.85               29.00,30.85 24.00,30.85 24.00,30.85               24.00,30.85 0.00,39.00 0.00,39.00               0.00,39.00 0.00,0.00 0.00,0.00               0.00,0.00 60.00,0.00 60.00,0.00 Z" />  </svg>

My goal for execution time for this OpenCV procedure on my Pi Zero is about 1-2 seconds or less (currently taking ~0.18 secs).

I have cobbled together something that sortof results in the sameish points that are in the Gimp xml file. I am not sure at all if it is doing what Gimp does with regard to the hue range of the mask. I have not yet figured out how to apply the minimum radius on the mask, I am pretty sure I will need that when the mask gets a 'grass' clump on the edge of the hard surface as part of the mask. Here are all the contour points so far (ptscanvas.bmp):

As of 7/6/2018 5:08 pm EST, here is the 'still messy' script that sortof works and found those points;

import numpy as np  import time, sys, cv2    img = cv2.imread('2-60.JPG')  cv2.imshow('Original',img)  # get a blank pntscanvas for drawing points on   pntscanvas = np.zeros(img.shape, np.uint8)    print (sys.version)    if sys.version_info[0] < 3:      raise Exception("Python 3 or a more recent version is required.")    def doredo():      start_time = time.time()        # Use kmeans to convert to 2 color image      hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)      Z = hsv_img.reshape((-1,3))      Z = np.float32(Z)      # define criteria, number of clusters(K)       criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)      K = 2      ret,label,center=cv2.kmeans(Z,K,None,criteria,10,cv2.KMEANS_RANDOM_CENTERS)        # Create a mask by selecting a hue range around the lowest hue of the 2 colors      if center[0,0] < center[1,0]:          hueofinterest = center[0,0]      else:          hueofinterest = center[1,0]      hsvdelta = 8      lowv = np.array([hueofinterest - hsvdelta, 0, 0])      higv = np.array([hueofinterest + hsvdelta, 255, 255])      mask = cv2.inRange(hsv_img, lowv, higv)        # Extract contours from the mask      ret,thresh = cv2.threshold(mask,250,255,cv2.THRESH_BINARY_INV)      im2,contours,hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)            # Find the biggest area contour      cnt = contours[0]      max_area = cv2.contourArea(cnt)        for cont in contours:          if cv2.contourArea(cont) > max_area:              cnt = cont              max_area = cv2.contourArea(cont)        # Make array of all edge points of the largets contour, named allpnts        perimeter = cv2.arcLength(cnt,True)      epsilon = 0.01*cv2.arcLength(cnt,True) # 0.0125*cv2.arcLength(cnt,True) seems to work better      allpnts = cv2.approxPolyDP(cnt,epsilon,True)            end_time = time.time()      print("Elapsed cv2 time was %g seconds" % (end_time - start_time))        # Convert back into uint8, and make 2 color image for saving and showing      center = np.uint8(center)      res = center[label.flatten()]      res2 = res.reshape((hsv_img.shape))        # Save, show and print stuff      cv2.drawContours(pntscanvas, allpnts, -1, (0, 0, 255), 2)      cv2.imwrite("pntscanvas.bmp", pntscanvas)      cv2.imshow("pntscanvas.bmp", pntscanvas)      print('allpnts')      print(allpnts)      print("center")      print(center)      print('lowv',lowv)      print('higv',higv)      cv2.imwrite('mask.bmp',mask)      cv2.imshow('mask.bmp',mask)      cv2.imwrite('CvKmeans2Color.bmp',res2)      cv2.imshow('CvKmeans2Color.bmp',res2)    print ("Waiting for 'Spacebar' to Do/Redo OR 'Esc' to Exit")  while(1):      ch = cv2.waitKey(50)      if ch == 27:          break      if ch == ord(' '):          doredo()            cv2.destroyAllWindows()

Left to do:

1. Add mask radiusing on non-edge pixels to take care of raw masks like this one that Gimp creates before it runs a min radius on the mask:

1a. EDIT: As of July 9, 2018, I have been concentrating on this issue as it seems to be my biggest problem. I am unable to have cv2.findcontours smooth out the 'edge grass' as well as Gimp does with its magic wand radius feature. Here on the left, is a 2 colour 'problem' mask and the overlaid resultant 'Red' points that are found directly using cv2.findcontours and on the right, the Gimp radiused mask applied to the left images 'problem' mask before cv2.findcontours is applied to it, resulting in the right image and points:

I have tried looking at Gimps source code but it is way beyond my comprehension and I can not find any OpenCV routines that can do this. Is there a way to apply a minimum radius smoothing to the 'non-edge' pixels of an edge mask in OpenCV??? By 'non-edge' I mean that as you can see Gimp does not radius these 'corners' (inside Yellow highlight) but only seems to apply the radius smoothing to edges 'inside' the image (Note: Gimps radiusing algorithm eliminates all the small islands in the mask which means that you don't have to find the largest area contour after cv2.findcontours is applied to get the points of interest):

2. Remove irrelevant array points from allpnts that are on the image edge.
3. Figure out why the array points that it finds seem to border around the green grass instead of the hard surface, I thought I was working with the hard surface hue.
4. Figure out why the hard surface color in CvKmeans2Color.bmp appears orange and not beige as in Gimps conversion AND why doesn't this match pixel for pixel with Gimps conversion? Here is CvKmeans2Color.bmp and Gimps:

EDIT: As of 5pm EST July 12, 2018: I have resorted to the language I can most easily create code with, VB6, ughh, I know. Anyway I have been able to make a line/edge smoothing routine that works on the pixel level to do the min radius mask I want. It works like a PacMan roaming along the right side of an edge as close at it can and leaves behind a breadcrumb trail on the Pac's left side. Not sure I can make a python script from that code but at least I have a place to start as nobody has confirmed that there is an OpenCV alternative way to do it. If anyone is interested here is a compiled .exe file that should run on most windows systems without an install (I think). Here is a screenshot from it (Blue/GreenyBlue pixels are the unsmoothed edge and Green/GreenyBlue pixels are the radiused edge):

You can get the gist of my process logic by this VB6 routine:

Sub BeginFollowingEdgePixel()     Dim lastwasend As Integer     wasinside = False     While (1)        If HitFrontBumper Then           GoTo Hit        Else           Call MoveForward        End If        If circr = orgpos(0) And circc = orgpos(1) Then           orgpixr = -1 'resets Start/Next button to begin at first first found blue edge pixel           GoTo outnow 'this condition indicates that you have followed all blue edge pixels        End If        Call PaintUnderFrontBumperWhite        Call PaintGreenOutsideLeftBumper  nomove:        If NoLeftBumperContact Then           Call MoveLeft           Call PaintUnderLeftBumperWhite           Call PaintGreenOutsideLeftBumper           If NoLeftBumperContact Then              If BackBumperContact Then                 Call MakeLeftTheNewForward              End If           End If        ElseIf HitFrontBumper Then  Hit:           Call PaintAheadOfForwardBumperGreen           Call PaintGreenOutsideLeftSide           Call MakeRightTheNewForward           GoTo nomove        Else           Call PaintAheadOfForwardBumperGreen           Call PaintGreenOutsideLeftSide           Call PaintUnderFrontBumperWhite        End If        If (circr = 19 + circrad Or circr = -circrad Or circc = 19 + circrad Or circc = -circrad) Then           If lastwasend = 0 And wasinside = True Then              'finished following one edge pixel              lastwasend = 1              GoTo outnow              Call redrawit           End If        Else           If IsCircleInsideImage Then              wasinside = True           End If           lastwasend = 0        End If        Pause (pausev) 'seconds between moves - Pressing Esc advances early     Wend  outnow:  End Sub

1 Answers

Answers 1

Okay, I finally had time to look at this. I will address each point of yours and then show the changes in the code. Let me know if you have any questions, or suggestions.

1. Looks like you were able to do this yourself well enough.

   1.a. This can be taken care of by blurring the image before doing any processing to it. The following changes to the code were made to accomplish this;

   ... start_time = time.time()                                                blur_img = cv2.GaussianBlur(img,(5,5),0) #here                          # Use kmeans to convert to 2 color image                               hsv_img = cv2.cvtColor(blur_img, cv2.COLOR_BGR2HSV) ... 

2. I have changed the code to remove points that are on a line that perfectly follows the side of the image. It should be basically impossible for a grass edge to also coincide with this.

   ... allpnts = cv2.approxPolyDP(cnt,epsilon,True)                            new_allpnts = []                                                         for i in range(len(allpnts)):                                              a = (i-1) % len(allpnts)                                               b = (i+1) % len(allpnts)                                                if ((allpnts[i,0,0] == 0 or allpnts[i,0,0] == (img.shape[1]-1)) and (allpnts[i,0,1] == 0 or allpnts[i,0,1] == (img.shape[0]-1))):                   tmp1 = allpnts[a,0] - allpnts[i,0]                                     tmp2 = allpnts[b,0] - allpnts[i,0]                                                                                                                              if not (0 in tmp1 and 0 in tmp2):                                          new_allpnts.append(allpnts[i])     else:         new_allpnts.append(allpnts[i]) ... cv2.drawContours(pntscanvas, new_allpnts, -1, (0, 0, 255), 2) ... 

3. Due to how the contours are found in the image, we can simply flip the thresholding function and find the contour around the other part of the image. Changes are below;

   ... #Extract contours from the mask                                       ret,thresh = cv2.threshold(mask,250,255,cv2.THRESH_BINARY) #here       im2,contours,hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) ... 

4. As for the color differences, you have converted your image into HSV format and before saving you are not switching it back to BGR. This change to HSV does give you better results so I would keep it, but it is a different palette. Changes are below;

   ... cv2.imshow('mask.bmp',mask)                                            res2 = cv2.cvtColor(res2, cv2.COLOR_HSV2BGR)                           cv2.imwrite('CvKmeans2Color.bmp',res2)                                 cv2.imshow('CvKmeans2Color.bmp',res2) ... 

Disclaimer: These changes are based off of the python code from above. Any changes to the python code that are not in the provide code my render my changes ineffective.

Read More

Ways to combine two deep learning based classifiers

I want to have a primary CNN based classifier and a similar secondary classifier for image regions.

Both classifiers will be used on image regions. I need the first classifier to be used on a primary region while the secondary classifier to be used on assistive regions and will be used to support the decision made by the first classifier with further evidence.

Thus the primary image region and the assistive ones will be used to infer one class label at a time.

What other ways or architectures exist these days to perform such a task, instead of ROI Pooling?

Ideally, I would like to have a classifier scheme similar to the one of this paper but without the use of ROI Pooling.

https://arxiv.org/pdf/1505.01197.pdf

2 Answers

Answers 1

Yaw Lin's answer contains a good amount of information, I'll just build on what he said in his last paragraph. I think the essence of what you want to do is not so much to process the person and the background independently and compare the results (that's clearly what you said you're doing), but to process the background first and infer from it the kinds of expectations you have for the primary region. Once you have some expectations, you can compare the primary region to the most significant expectations.

For example, from Figure 1 (b) in your Arxiv link, if you can process the background and determine that it's outdoors in a highly populated region, then you can focus a lot of the probability density function of what the person is doing in social outdoor activities, making jogging much more likely as a guess before you even process the figure you're interested in. In contrast, for Figure 1 (a), if you can process the background and tell that it's indoors and contains computers, then you can focus probability on lone indoor computer-based activities, skyrocketing the probability of "working on a computer".

Answers 2

You can take a look at this https://arxiv.org/pdf/1611.10012.pdf which contains comprehensive survey of recent detection architectures. Basically there are 3 meta architectures and all models fall into one of those categories:

1. Faster-RCNN: Similar to the paper you have referenced, this is the improved version of fast-rcnn which did not use selective search and directly integrate proposal generation into the network known as region proposal network(rpn).
2. RFCN:similar in architecture to 1, except that roi pooling is performed differently, known as position sensitive roi pooling.
3. SSD:Modifies the rpn in Faster-rcnn to directly output class probabilities, eliminating the need for per-roi computation as is done in roi pooling. This is the fastest architecture type. Yolo falls into this architecture.

I think based on my rough read through of the paper you have referenced, type 3 is the one you are looking for. However, in terms of implementation, it can be a little tricky to implement equation 3, i.e. you may need to stop backpropagating gradients to regions(or at least think about how it could effect the final results) that do not overlap with primary region as this architecture type computes probabilities for whole image.

I also note that there is in fact no primary/secondary "classifiers". The paper described primary/secondary "regions", the primary region is the region that contains the person(i.e use a person detector to find primary region first). And secondary regions are those that overlap with primary region. For activity classification there is only one classifier except that primary region carries more weight and secondary regions each contribute a little to the final prediction score.

Read More

Tensorflow Training and Validation Input Queue Separation

I tried to replicate the Fully Convolutional Network results using TensorFlow. I used Marvin Teichmann's implementation from github. I only need to write the training wrapper. I create two graphs that share variables and two input queues, one for training and one for validation. To test my training wrapper, I used two short lists of training and validation files and I do a validation immediately after every training epoch. I also printed out the shape of every image from the input queue to check whether I get the correct input. However, after I started the training, it seems that only the images from the training queue is being dequeued. So both the training and validation graphs take input from the training queue and the validation queue is never accessed. Can anyone help explain and solve this problem?

Here's part of the relevant code:

def get_data(image_name_list, num_epochs, scope_name, num_class = NUM_CLASS):     with tf.variable_scope(scope_name) as scope:         images_path = [os.path.join(DATASET_DIR, i+'.jpg') for i in image_name_list]         gts_path = [os.path.join(GT_DIR, i+'.png') for i in image_name_list]         seed = random.randint(0, 2147483647)         image_name_queue = tf.train.string_input_producer(images_path, num_epochs=num_epochs, shuffle=False, seed = seed)         gt_name_queue = tf.train.string_input_producer(gts_path, num_epochs=num_epochs, shuffle=False, seed = seed)         reader = tf.WholeFileReader()         image_key, image_value = reader.read(image_name_queue)         my_image = tf.image.decode_jpeg(image_value)         my_image = tf.cast(my_image, tf.float32)         my_image = tf.expand_dims(my_image, 0)         gt_key, gt_value = reader.read(gt_name_queue)         # gt stands for ground truth         my_gt = tf.cast(tf.image.decode_png(gt_value, channels = 1), tf.float32)         my_gt = tf.one_hot(tf.cast(my_gt, tf.int32), NUM_CLASS)         return my_image, my_gt  train_image, train_gt = get_data(train_files, NUM_EPOCH, 'training') val_image, val_gt = get_data(val_files, NUM_EPOCH, 'validation') with tf.variable_scope('FCN16') as scope:         train_vgg16_fcn = fcn16_vgg.FCN16VGG()         train_vgg16_fcn.build(train_image, train=True, num_classes=NUM_CLASS, keep_prob = KEEP_PROB)         scope.reuse_variables()         val_vgg16_fcn = fcn16_vgg.FCN16VGG()         val_vgg16_fcn.build(val_image, train=False, num_classes=NUM_CLASS, keep_prob = 1) """ Define the loss, evaluation metric, summary, saver in the computation graph. Initialize variables and start a session. """ for epoch in range(starting_epoch, NUM_EPOCH):     for i in range(train_num):         _, loss_value, shape = sess.run([train_op, train_entropy_loss, tf.shape(train_image)])         print shape     for i in range(val_num):         loss_value, shape = sess.run([val_entropy_loss, tf.shape(val_image)])         print shape 

1 Answers

Answers 1

To make sure you are reading different images you could run:

[train_image_np, val_image_np] = sess.run([train_image, val_image]) 

To reuse variables this is better and safer:

with tf.variable_scope('FCN16') as scope:    train_vgg16_fcn = fcn16_vgg.FCN16VGG()      train_vgg16_fcn.build(train_image, train=True, num_classes=NUM_CLASS, keep_prob = KEEP_PROB) with tf.variable_scope(scope, reuse=True):    val_vgg16_fcn = fcn16_vgg.FCN16VGG()    val_vgg16_fcn.build(val_image, train=False, num_classes=NUM_CLASS, keep_prob = 1) 

Read More

Edge following with camera

I want to follow the rightmost edge in the following picture with a line following robot.

I tried simple "thresholding", but unfortunately, it includes the blurry white halo:

The reason I threshold is to obtain a clean line from the Sobel edge detector:

Is there a good algorithm which I can use to isolate this edge/move along this edge? The one I am using currently seems error prone, but it's the best one I've been able to figure out so far.

Note: The edge may curve or be aligned in any direction, but a point on the edge will always lie very close to the center of the image. Here's a video of what I'm trying to do. It doesn't follow the edge after (1:35) properly due to the halo screwing up the thresholding.

---

Here's another sample:

Here, I floodfill the centermost edge to separate it from the little bump in the bottom right corner:

1 Answers

Answers 1

Simplest method (vertical line)

If you know that your image will have black on the right side of the line, here's a simple method:

1) apply the Sobel operator to find the first derivative in the x direction. The result will be an image that is most negative where your gradient is strongest. (Use a large kernel size to average out the halo effect. You can even apply a Gaussian blur to the image first, to get even more averaging if the 7x7 kernel isn't enough.)

2) For each row of the image, find the index of the minimum (i.e. most negative) value. That's your estimate of line position in that row.

3) Do whatever you want with that. (Maybe take the median of those line positions, on the top half and the bottom half of the image, to get an estimate of 2 points that describe the line.)

Slightly more advanced (arbitrary line)

Use this if you don't know the direction of the line, but you do know that it's straight enough that you can approximate it with a straight line.

1)

dx = cv2.Sobel(grayscaleImg,cv2.cv.CV_32F,1,0,ksize=7) dy = cv2.Sobel(grayscaleImg,cv2.cv.CV_32F,0,1,ksize=7) angle = np.atan2(dy,dx) magnitudeSquared = np.square(dx)+np.square(dy) 

You now have the angle (in radians) and magnitude of the gradient at each point in your image.

2) From here you can use basic numpy operations to find the line: Filter the points to only keep points where magnitudeSquared > some threshold. Then grab the most common angle (np.bincount() is useful for that). Now you know your line's angle.

3) Further filter the points to only keep points that are close to that angle. You now have all the points on your line. Fit a line through the coordinates of those points.

Most advanced and brittle (arbitrary curve)

If you really need to handle a curve, here's one way:

1) Use your method above to threshold the image. Manually tune the threshold until the white/black division happens roughly where you want it. (Probably 127 is not the right threshold. But if your lighting conditions are consistent, you might be able to find a threshold that works. Confirm it works across multiple images.)

2) Use OpenCV's findcontours() to fit a curve to the white/black boundary. If it's too choppy, use approxPolyDP() to simplify it.

Read More

Building a simple image search using TensorFlow

I need to implement a simple image search in my app using TensorFlow. The requirements are these:

1. The dataset contains around a million images, all of the same size, each containing one unique object and only that object.
2. The search parameter is an image taken with a phone camera of some object that is potentially in the dataset.

I've managed to extract the image from the camera picture and straighten it to rectangular form and as a result, a reverse-search image indexer like TinEye was able to find a match.

Now I want to reproduce that indexer by using TensorFlow to create a model based on my data-set (make each image's file name a unique index).

Could anyone point me to tutorials/code that would explain how to achieve such thing without diving too much into computer vision terminology?

Much appreciated!

1 Answers

Answers 1

The Wikipedia article on TinEye says that Perceptual Hashing will yield results similar to TinEye's. They reference this detailed description of the algorithm. But TinEye refuses to comment.

---

The biggest issue with the Perceptual Hashing approach is that while it's efficient for identifying the same image (subject to skews, contrast changes, etc.), it's not great at identifying a completely different image of the same object (e.g. the front of a car vs. the side of a car).

TensorFlow has great support for deep neural nets which might give you better results. Here's a high level description of how you might use a deep neural net in TensorFlow to solve this problem:

Start with a pre-trained NN (such as GoogLeNet) or train one yourself on a dataset like ImageNet. Now we're given a new picture we're trying to identify. Feed that into the NN. Look at the activations of a fairly deep layer in the NN. This vector of activations is like a 'fingerprint' for the image. Find the picture in your database with the closest fingerprint. If it's sufficiently close, it's probably the same object.

The intuition behind this approach is that unlike Perceptual Hashing, the NN is building up a high-level representation of the image including identifying edges, shapes, and important colors. For example, the fingerprint of an apple might include information about its circular shape, red color, and even its small stem.

---

You could also try something like this 2012 paper on image retrieval which uses a slew of hand-picked features such as SIFT, regional color moments and object contour fragments. This is probably a lot more work and it's not what TensorFlow is best at.

Read More

Calculation of new homography from existing homography values

I have 4 thin pillars on the ground and I have put red dots on all the pillars above 1m, 2m and 3 meter. As in below image

And the coordinates in the real world coordinates of the bottom corners (where pillars touches ground are) are A1, A2, A3, A4

And I have taken image of the above structure. And in image, the real world coordinate, red dot above 1m are mapped to I1, I2, I3, I4 in the image. Using the coordinates A1, A2, A3, A4 and I1, I2, I3, I4 I have calculated homography matrix H1

Similarly Red dot above 2m are mapped to I5, I6, I7, I8 in the image and using the points A1, A2, A3, A4 and I5, I6, I7, I8 I have calculated homography matrix H2

Similarly Red dot above 3m are mapped to I9, I10, I11, I12 in the image and using the poits A1, A2, A3, A4 and I9, I10, I11, I12 I have calculated homography matrix H3

Now Using above information, I need to find the homography matrix between A1, A2, A3, A4 and Iu1, Iu2, Iu3, Iu4, where Iu1, Iu2, Iu3, Iu4 are image coordinates(unknown) if red dots were on the pillars, 4m above the ground.

How can I find this ?

0 Answers

Read More

Extracting attributes from images using Scikit-image

I've been using scikit-image to classify road features with some success. See below: . I am having trouble doing the next step which is to classify the features. For example, let's say these features are located in the box (600, 800) and (1400, 600).

The code I'm using to extract the information is:

from skimage import io, segmentation as seg color_image = io.imread(img)   plt.rcParams['image.cmap'] = 'spectral' labels = seg.slic(color_image, n_segments=6, compactness=4) 

The objective is to have a table in the following form:

Image, feature_type, starting_pixel, ending_pixel 001    a             (600, 600),     (1300, 700)  002    b             (600, 600),     (1100, 700) 002    undefined     (700, 700),     (900, 800) 

feature_type would be based on colours, ideally shoulders would be one colour, trees and brush would be another, etc.

How can I extract the data I need? (i.e: have scikit break the image into different components where I know the location of each component. I can then pass each component to a classifier which will identify what each component is) Thanks!

1 Answers

Answers 1

It is the first time I try that package.. I tried with a simpler image and I get more or less the right results:

from skimage import io, segmentation as seg import matplotlib as plt import numpy as np color_image = io.imread('smallimg.jpg') labels = seg.slic(color_image, n_segments=4, compactness=4) for section in np.unique(labels):     rows, cols = np.where(labels == section)     print("Image="+str(section))     print("Top-Left pixel = {},{}".format(min(rows), min(cols)))     print("Bottom-Right pixel = {},{}".format(max(rows), max(cols)))     print("---") 

Output:

Image=0 Top-Left pixel = 3,1 Bottom-Right pixel = 15,18 --- Image=1 Top-Left pixel = 26,1 Bottom-Right pixel = 34,18 --- Image=2 Top-Left pixel = 43,1 Bottom-Right pixel = 52,16 --- Image=3 Top-Left pixel = 0,0 Bottom-Right pixel = 59,19 --- 

Notice that the right-most pixel is not precisely what I mean because of the gradient. The last segment is the white background.

I tried with your image but I think you will have to get the segmentation right. I would use n_segments=7 if you want to get the 6 images + background.

I also see in the documentation about compactness: "This parameter depends strongly on image contrast and on the shapes of objects in the image.". So what you want could be difficult to achieve.

If you are plotting the six pictures on the image you show above, why don't you get those coordinates when you plot the pictures instead of applying segmentation to the end result?

Read More

Pyramidal Histogram Of Oriented Gradients - Trilinear interpolation

Hello im struggling with an implementation of this article: https://goo.gl/8mpIuq

I performed bilinear interpolation over the histogram bins and the results are better with this interpolation, however on page 2 its also mentioned that a trilinear interpolation is added when the pyramid level reaches level 2. I have read this answer HOG Trilinear Interpolation of Histogram Bins and I completely understand the formula behind trilinear interpolation over 2x2 block sizes, but in this article we have a 3x3 block size and 7x7 on pyramid level 3, because these block sizes yield the best results.

The main point about trilinear interpolation is that each pixel in a cell contributes to its local cell by a weight which is defined as the position in each block. I don't know how to represent the location of a pixel in 3x3 block size or what kind of formula should i use.

Thank you for all your help!

EDIT: Another explanation with 2x2 block size http://pep.ijieee.org.in/journal_pdf/11-126-142960909718-22.pdf

0 Answers

Read More