Search engines should be able to "Subscribe" to web sites in order to be automatically informed of the latest changes.

Think about some kind of "reversed RSS" and the idea that it is sometime better to push than pull...

Isn't that the whole idea of Google Sitemaps?

Not exactly, the publish-subscribe mechanism is dynamic, this is more an action than an information. When there is a change somewhere on a website, the search engine could be "instantly" informed for a real time indexation.

Is that practical though? Google and its peers prioritise how often they crawl a page based upon its relative importance to the web as a whole.

If a modest site tells the big 4 that one or more of its pages have been updated, the reality is that they're not going to devote processing power to it ahead of sites they deem to be more important .

Are you claiming that real time search engines are useless? I don't understand how a website can be qualified more important than an other. At the beginning, any new web site (or new page) are unknown, no link, no PageRank. A big thing can come from anywhere...

I'm not claiming anything, but is it not the case that there are some sites that Google indexes more than once a day whilst it only refreshes its knowledge of minor sites once every few weeks?

This is just sheer pragmatism on their part. Popular sites with millions of visitors per day that generate large amounts of highly valued content warrant more regular attention than sites that don't.

Even Google has finite processing power to devote to its index.

PS: If you know different, I'd be interested to learn.
:)

Websites should be able to "push" changes to some kind of "datahubs". Datahubs receive updates and redistribute them to other datahubs in a decentralized way. Datahubs are both client and server and they are linked together with win-win links: I send you my data and you give me your. Datahubs can index the data on the way and potentially become search engines 100 time cheaper than Google.