“Crickhowell Castle, also known as Alisby’s castle, is a conspicuous feature of the small market town and occupies a vantage point with commanding views along the Usk valley. It began life as a motte and bailey with timber buildings, probably built by the Turberville family in the 12th century. In 1272 it was rebuilt in stone, still to the basic plan, by Sir Grimbald Pauncefote, who married Sybil, a Turberville heiress.” http://www.castlewales.com/crickhwl.html
“The castle as you enter the town from the east can be seen from the A40 road, which is on the route of an old Roman road from London to Fishguard.
Alisby, who was a knight serving Roger Mortimer (a Norman), once held the castle in its early days and the castle is named Alisby Castle after him. The Turbervilles held the castle in the 13th Century, but Lady Sybil Turberville, heiress to the estate, married Sir Grimbald Pauncefote and this resulted in the Pauncefote’s running the castle.
Reputed to be built in the 12th century, the castle was attacked in 1403 by Owain Glyndwr (true name Owain ap Gruffydd, son of Gruffydd Fychan, descendant of Bleddyn ap Cynfyn, Prince of Powys) and was believed to have been abandoned in the same century.
The collapse of the structure and robbing of the stone by locals to build most of the surrounding properties has left only the remains of two towers, one circular and one rectangular. Part of the rectangular tower collapsed during the First World War.” http://www.bbc.co.uk/wales/mid/sites/crickhowell/pages/john_addis_castle.shtml