These crawls are part of an effort to archive pages as they are created and archive the pages that they refer to. That way, as the pages that are referenced are changed or taken from the web, a link to the version that was live when the page was written will be preserved.
Then the Internet Archive hopes that references to these archived pages will be put in place of a link that would be otherwise be broken, or a companion link to allow people to see what was originally intended by a page's authors.
The board ousted Antony Jenkins despite one of the best half-year results in a while. And it’s not clear what will change. So why did the share price leap?
By dismissing the chief executive – who was endorsed by the Archbishop of Canterbury for his moral compass – the board is re-entering dangerous territory
The ousting of Barclays’ chief executive appears to send a clear message: banks are back to business as usual and genuine reform is as far away as ever
Nils Pratley: half-year results suggest bits investors tend to like are performing better, but whether healthy investment banking core emerges remains to be seen
Business leader: The allegations of wrongdoing in the bank's 'dark pool' trading system are bad news for Antony Jenkins, for they cannot all be laid at the feet of the old regime
Observer business agenda It’s H for hot air at Halfords