This might be worth trying. You could try to dig a tunnel underground, light a fire under the castle walls and hope that the walls weaken and fall down. You could also try attacking that wall with catapults. But if those inside the castle realise what you are doing they may also dig a tunnel and attack you underground.
Back to the game!