Saturday, 22 November 2014

Making Faces and Faking Faces

I have been playing with the Haar-cascade-based face detector demo that comes with OpenCV. Among other things the demo can, in real time, detect faces in video frames from a camera. It draws circles around whatever it finds.

It occurred to me to confirm that the demo will detect multiple faces simultaneously. As I only have one face of my own, this doubled as an experiment to see how much of a realistic face I had to draw before the detector would recognise it as a face.

Loosely speaking, a Haar cascade models the thing to be detected as an arrangement of contrasting light and dark rectangular regions. Unsurprisingly, then, it didn't pick up a very stylised face with just a line for a mouth, dots for eyes and no nose. But once I'd darkened the mouth line enough, and put in some eyebrows and some nostrils, the face detector started to spot the fake face quite reliably.

The demo also does a second detection step within any region found by the first step. This can be used to find the eyes within the face, for example. The eye detector proved somewhat harder to fool. Even with heavy ellipses for the eyes, and well defined irises and pupils, it wasn't convinced. Only once I started to sketch the shadow either side of the nose did it become likely that the eye detector would detect eyes in my fake face.

Face detector draws circles around the regions it detects as containing faces and eyes
The face I drew is nonetheless far from realistic. Any face detector that finds nearly all things that are faces will also find some things that are not faces. In machines, this explains why Google Street View blurs out KFC's Colonel Sanders. In humans, we call it pareidolia.

Saturday, 2 August 2014

Time Passes

A year after Mental Block returned to the UK, I'm starting to explore ideas for what the next project might look like. I dug out a bit of code that I hadn't touched since 2012 to draw a Penrose tiling (the kite-and-dart tiling, specifically) and tidied it up a bit, thinking it might be interesting to make a jigsaw puzzle whose pieces form that tiling. But first I hacked it a bit to produce a Facebook cover picture:

(The method for producing the decaying effect here is mildly interesting. I wanted it to be probabilistic but deterministic. My answer was to hash each tile's position and map the hash onto the [0-1] interval; if the result is greater than the probability density at that point then the tile is not drawn. This could equally have been achieved using a pseudo-random number generator, but it has the nice property of being a pure function of the tile, so no state needs to be carried around – a particular advantage when writing in a pure functional language.)

As for the jigsaw puzzle, I had begun to have my doubts about its suitability for a Nowhere-style project. One of the things that kept people coming back to Mental Block was that it's open-ended and doesn't have a right answer. The best things that people built with it were the ones which I had never imagined and that didn't seem possible:

Similarly, at Nowhere in 2014, one of the best things was the noticeboard:

Again, this is something with no finished state. Consequently, it was rewritten hundreds of times over the course of the event.

A jigsaw puzzle, by contrast, would probably be solved once and then left alone. How can it be given a sense of restlessness and open-endedness? One way would be to paint a different design on each side, so that if one side is finished then the other isn't. But that's still only two possibilities, and doesn't leave much room for anyone else's imagination.

What if the puzzle is circular, and its two sides link up with each other like a Möbius strip? Then, if you start assembling the bottom left of it, you will find a mismatch once you work your way across to the top right. Turning a nearby piece over, you find that one of its faces matches the discontinuity in one direction, and its other face matches the discontinuity in the other direction. As you turn more pieces over, you push the mismatch somewhere else but you can never eliminate it.

Simplifying from a jigsaw puzzle to an array of 60 sectors, this idea becomes a clock. Each sector has two faces; one face matches its clockwise neighbour and the other matches its anticlockwise neighbour. There's always a mismatch, and that's how you tell the time. Here's a rough draft of that idea, built from a 24 hour time lapse video of Barcelona:

The idea works best at the top where night and day are obviously different, and worst at the bottom where the dawn light and the dusk light aren't much different from each other.

I also experimented with a simple square grid, warped so that the discontinuity is evident in the angle of the squares:

At certain angles this is hard to read. That can be helped by modulating the hue. If each full revolution on the clock covers half the colour wheel, then the discontinuity will always show complementary colours and will be very easy to see:

The next step is obviously to relieve the tedium of squares with the help of Mr. Escher's work:

I'm not sure where this is going, now. Any thoughts?