Pete from the USA has got it right for this country, too. Anyone with a computer or typewriter can knock up genuine-looking headed paper and an invoice. As the authorities are aware of that it is, in itself, proof of nothing.
Any kind of statement, receipt or invoice, including a Paypal one, even a handwritten one, is good. It is even better backed up by an Inland Revenue or VAT trusted source. Banks and credit card companies are trusted sources as far as they are concerned - no, guys, what we think is another matter for another debate
.
Proof of cash movements reflecting the amounts invoiced and, roughly, dates of payment is adequate evidence. Your bank or credit card statement provides that.
It may be that you will never get an inspection by the authorities, but it would be foolish to throw away bank and credit card statements which show business transactions, even if you do have invoices and all the other paperwork.