Annotate an image in UWP

A while ago I created a tool for the HoloLens device to calculate the interpupilary distance (distance between your eyes pupils) to properly configure the device. You can read all the details about that on the GitHub page.

In summary, I used the Microsoft Cognitive Services Face Detection API to detect facial landmarks and calculate the distance based on them.
One of the most interesting parts of the app was to load an image and annotate it with the locations of the faces detected in the image, just like you see in the online demos.

The solution I chose was to use a custom Canvas overlaid on top of the image. Knowing the actual dimensions of the image and the actual size when rendered, allows to calculate the position and size of the face rectangles as returned by the Face API. Once those ratios are calculated, we draw each rectangle (and any other landmarks we need) making sure we refresh every time the size changes since that would cause the ratios between the image size and render size to change.

Full code below:

public class AnnotationCanvas : Canvas
{	
	public AnnotationCanvas()
	{
		SizeChanged += OnSizeChanged;
		App.EventAggregator.GetEvent<AnnotationCanvasUpdateEvent>().Subscribe(UpdateAnnotations);
	}

	public List<FaceDescriptor> Faces
	{
		get { return (List<FaceDescriptor>)GetValue(FacesProperty); }
		set { SetValue(FacesProperty, value); }
	}

	public static readonly DependencyProperty FacesProperty = DependencyProperty.Register("Faces", typeof(List<FaceDescriptor>), typeof(AnnotationCanvas), new PropertyMetadata(null));


	public Size ImageRealSize
	{
		get { return (Size)GetValue(ImageRealSizeProperty); }
		set { SetValue(ImageRealSizeProperty, value); }
	}

	public static readonly DependencyProperty ImageRealSizeProperty = DependencyProperty.Register("ImageRealSize", typeof(Size), typeof(AnnotationCanvas), new PropertyMetadata(null));

	public void UpdateAnnotations()
	{
		Children.Clear();

		if (Faces == null || ImageRealSize == null)
			return;

		var ar = ImageRealSize.Height / this.ActualHeight;
		var replacedFaces = Faces.Select(f => new Rect(f.FaceBounds.Left / ar, f.FaceBounds.Top / ar, f.FaceBounds.Width / ar, f.FaceBounds.Height / ar)).ToArray();


		var faceFrames = replacedFaces.Select(x => {
			var n = new Rectangle() { Width = x.Width, Height = x.Height };
			n.SetValue(Canvas.LeftProperty, x.Left);
			n.SetValue(Canvas.TopProperty, x.Top);
			n.Stroke = new SolidColorBrush(Colors.Red);
			n.StrokeThickness = 3;
			return n;
		});

		foreach (var faceFrame in faceFrames)
			Children.Add(faceFrame);
	}

	private void OnSizeChanged(object sender, SizeChangedEventArgs e)
	{
		UpdateAnnotations(false);
	}
}

Here’s a screenshot of it in action (with an extra adornment for the HoloLens IPD app)

Some obvious improvements would be ability to configure color, strokes and potentially support more generic objects. I am also considering embedding the image directly in the canvas and make it a bit more encapsulated but for now it works fine as it is.