You need to file a VAT return, if you are saying you've made no sales, then file a Nil VAT return. You file that through bridging software or digital accounting software.
You will not be allowed to deregister whilst you have an outstanding VAT return.
The £2,000 is not a laughable made up figure, it is based on what you put on your VAT registration when the question asked you what level of turnover you expect to have in the next 12 months. If you said £50k of sales, output tax would be £10k and HMRC use a formula to estimate input tax that you might reclaim each quarter and arrived at the £2k figure, it is an estimate based on your own turnover estimate and the sector you are in (retail, IT, consultancy, etc).
Once the return is filed, the estimated £2k will be adjusted to reflect the correct liability, in your case you are stating the liability is Nil, the £2k disappears, you are up to date with your filed VAT returns, no liability and no penalties or interest, then you can deregister, you can deregister via the government gateway/portal that you have for VAT.
HMRC can have delays in processing VAT registrations, especially overseas traders in Asia and as HMRC use the postal system to communicate, there can be further delays as post can take 10-14 days to arrive at the overseas destination.
Registering for VAT (or indeed any tax in a foreign jurisdiction) always comes with a risk of not knowing the procedures, not knowing the domestic legislation or rules and the risk of having to deal with problems like delays that have caused this issue which then need to be fixed.
When you received your VAT registration letter in the post, you should have promptly setup your government gateway and added the VAT service and you would have seen when your first VAT return was due and potentially avoided the late filing issue. There was clearly some delay in you receiving the letter from HMRC and this could have meant your first VAT return was due very quickly after the VAT number was issued, but equally, HMRC are not responsible for postal delays or if the post was lost/undelivered.
Stuff like this does happen, letters get lost or delayed in the post, HMRC will still issue estimated penalties as HMRC will not know you didn't receive the letter on time, etc. It is just part of dealing with a foreign tax office (HMRC) as a foreign trader. I have clients that have similar frustrations dealing with the French or German tax offices, it is just the way it is. There is no specific guidance that gives you a simple answer, instead you have to read various different parts of VAT guidance that deals with registration, penalties and deregistration, etc.