(Based on the article 'Pentaplexity' by Roger Penrose in Math. Intelligencer, 2 (1979) pp. 32-37)
One way to avoid rectangular symmetry is to consider radial symmetry, rather like spokes on a wheel radiating from a central hub. Penrose started with a pentagon and attempted to fill it with an infinitely repeatable series of subdivisions into smaller pentagons as shown:
First Subdivision
Second subdivision with the shaded section subdivided a third time
The second subdivision produces diamond shaped gaps and these grow 'spikes' at the third stage. Penrose noticed that this spiky diamond could be broken down into another pentagon, a pentagram and a 'boat':
When these new shapes grow spikes at the next subdivision, they too can be broken down into boats, pentagons and pentagrams and (this is the clever bit) no other new shapes :
Thanks to this 'no new shapes' property, Penrose reasoned that if we keep on subdividing and 'zoom in' further and further, the entire plane becomes tiled with only these four shapes (pentagons, diamonds, boats and pentagrams) and the tiling would necessarily be five-fold symmetric about the centre of the original pentagon.