I am developing a neural network based model to differentiate benign and malignant ovarian tumors. I am working with transvaginal ultrasound images, some of which contains yellow markers as seen in the first image below. Without going into too mach details, there is a risk that the presence of the markers could introduce a bias that the network will be able to exploit. As a way of trying to counter this bias, I will insert semi-random markers to the images during training and validation as a type of image augmentation.


During training, I import the images (stored as JPEG-files) and convert them to NumPy arrays.

  • Simply overlaying a marker is not good enough, since the marker would have a clear/sharp outline, while the "real"/already present markers are somewhat distorted due to anti-aliasing/JPEG-image compression.

I have some identical images with/without markers (e.g. first and second image below). After importing the two images as NumPy arrays the marker can be extracted by

  • Option 1: ... subtracting one image from the other (see the third and fourth image below for the result). (Would result in incorrect pixel values for the marker.)

  • Option 2: ... copying the image with markers and setting all cells to zero where the two images (with/without markers) are (almost) equal (see the fifth and sixth image below for the result).


Let's say, for simplicity, that the only thing that I wanted to achieve was to apply the extracted marker to a different transvaginal ultrasound image. How would I do that?

  • Simply overlaying the marker would create black patches around the marker.

  • Simply adding the pixel values would result in incorrect pixel values for the marker.

b) (given that a) is solved)

Let's say that I now also want to be able to insert a marker of a different size and orientation.

Orientation: I could rotate and move the marker. Size: Any suggestion of how I could go about doing this? I still want the marker to have the same properties, i.e. simply stretching is not an option.

from keras.preprocessing.image import load_img, img_to_array
from scipy.ndimage import rotate
import numpy as np

# importing images as NumPy arrays
img1 = img_to_array(load_img('img1.jpg'))
img2 = img_to_array(load_img('img2.jpg'))

# option 1
img3 = img2-img1   # extracting the marker by subtracting the image arrays
img3 = img3[:-100] # getting rid of the yellow/grey marker in the down-right corner

# option 2
img3 = img2[np.abs(img2 - img1) < 15] = 0

# rotating the image 
img3 = rotate(img3, angle=-23.6)

# cropping the image 
img3_copy = img3.copy()
img3_copy[np.abs(img3_copy) < 10**-4] = 0
idx = np.nonzero(img3_copy)
img3 = img3[np.min(idx[0]):np.max(idx[0])+1, np.min(idx[1]):np.max(idx[1])+1]

enter image description here enter image description here enter image description here enter image description here enter image description here enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.