I am trying to create markers on an image that would allow a user to select colors, mark features etc. Ultimately I would like to have the corresponding image pixel for further use via opencv.
I'm having a lot of trouble getting the expected color under the touch and it sometimes returns colors like magenta which are not even in the example image.
I am pretty sure the problem is with how I am converting the touch position to the values I am handing to the read_pixel function.
I have tried many different solutions without success so I think there is something I am missing here.
main.py
from kivy.app import App from kivy.properties import ListProperty, ObjectProperty from kivy.uix.image import AsyncImage from kivy.uix.relativelayout import RelativeLayout from kivy.uix.widget import Widget from kivy.uix.screenmanager import ScreenManager, Screen class Marker(Widget): selected_color = ListProperty([0,1,0]) def __init__(self, **kwargs): super(Marker, self).__init__(**kwargs) self.selected_pos = None def on_touch_down(self, touch): if self.collide_point(*touch.pos): print("Touched at Marker: {0}".format(touch.spos)) def on_touch_move(self, touch): self.set_position_from_touch(touch.spos) def set_position_from_touch(self, spos): # print("touch: {0}".format(touch)) self.image = self.parent.parent.image x = spos[0] * self.image.width y = spos[1] * self.image.height # setting position of the widget relative to touch self.pos = (x-self.width/2, y-self.height*(2/3)) # self.pos = (x, y) print("widget position : {0}".format(self.pos)) # converting widget position to pixel(row, column of selected_pixel = self.image.to_row_col(self.pos) print("selected Pixel: {0}".format(selected_pixel)) try: self.selected_color = self.image._coreimage.read_pixel( selected_pixel[0], selected_pixel[1]) # this skips conversion and just uses pos # self.pos[0], # self.pos[1]) except IndexError: print("position out of range") class MarkerManager(RelativeLayout): def __init__(self, **kwargs): super(MarkerManager, self).__init__(**kwargs) self.marker_mode = None self.features = [] def on_touch_down(self, touch): if self.collide_point(*touch.pos): child_touched = False print("Touched: {0}".format(touch)) if self.children: for child in self.children[:]: if child.collide_point(touch.pos[0], touch.pos[1]): child_touched = True child.dispatch('on_touch_down', touch) if not child_touched: print("Touched only Image at: {0}".format(touch.spos)) marker = Marker() self.features.append(marker) self.add_widget(marker) marker.set_position_from_touch(touch.spos) class SelectedImage(AsyncImage): def __init__(self, **kwargs): super(SelectedImage, self).__init__(**kwargs) self.allow_stretch=True self.keep_ratio=False def to_row_col(self, pos): pixels_x = self._coreimage.width pixels_y = self._coreimage.height pixel_x = (pos[0] / self.width) * pixels_x # pixel_y = (pos[1] / self.height) * self.pixels_y pixel_y = (1 - (pos[1] / self.height)) * pixels_y # should correspond to row column of image return [int(pixel_x), int(pixel_y)] class ImageScreen(Screen): image = ObjectProperty() manager = ObjectProperty() def __init__(self, **kwargs): super(ImageScreen, self).__init__(**kwargs) class PointsSelectorApp(App): def build(self): return ImageScreen() if __name__ == "__main__": PointsSelectorApp().run() pointsselector.kv
<ImageScreen>: image: image_id manager: manager_id SelectedImage: id: image_id source: "rainbow_checkerboard.jpg" keep_data: True MarkerManager: id: manager_id <Marker>: size_hint: None, None size: "40dp", "40dp" canvas: Color: rgb: self.selected_color Ellipse: pos: self.pos[0]+self.width/4, self.pos[1]+self.height/3 # pos: self.pos[0], self.pos[1] size: self.width*.6, self.height*.6 here is my image i have been using to test with "rainbow_checkerboard.jpg"
1 Answers
Answers 1
I believe this is a bug in the Kivy itself. Particularly I think that line 901 at the kivy/core/image/__init__.py
index = y * data.width * size + x * size raw = bytearray(data.data[index:index + size]) color = [c / 255.0 for c in raw] is wrong. It should be
index = y * data.rowlength + x * size instead.
Important thing here is that for performance reasons bitmap images in memory are alligned at 4 bytes addresses. Thus there is an explicit data.rowlength field. Usually that line works fine because images are typically well alligned so that data.rowlength = data.width * size. But your particular image is different: it uses 3-bytes RGB format (no alpha) and its width is odd 561 i.e. data.width * size = 1683 and so data.rowlength should be rounded up to 1684, which it actually is but that code doesn't take it into account. It means that often you read colors from two consecutive pixels and with RGB components randomly rotated.
Additionally since you use JPEG format for your image, borders between "cells" are not really strict. If you zoom in really hard, you can see some compression artifacts such as these from around bottom-left corner.
Those artifacts mutliplied by the aforementioned random color components rotation bug give you very strange (seemingly non-existent) colors.
Possible workarounds
- Change your image so that it is pre-aligned in the memory and the bug doesn't affect you (for example use mutliple of 4 as width)
- Submit a bug or even a pull request to the Kivy and wait for it to be fixed
- Patch
kivy/core/image/__init__.pyfile locally (I tried it and it seems to work fine).
0 comments:
Post a Comment