Hi Zahir Mohammed,
In order for relief of VAT to be applied on construction of a new building for charitable purpose, conditions need to be met.
Following guidance at section 14.7.4:
14. An explanation of dwellings, ‘relevant residential purpose’ and ‘relevant charitable purpose’
If these conditions are not met and the supply is made by a VAT registered business then VAT is applicable.
As you have stated your not registered for VAT and unable to recover, there is no other option of recovering this.
We can refer you to further guidance on how VAT effects charities here:
Goods or services supplied to charities (VAT Notice 701/58)
Thank you.