The only websites that should support password authentication are OpenID providers.

I agree even more at ‘And even that's debatable.’

I ONLY agree with "that's debatable".

Online banking services or other sensitive service providers should provide their own, more secure authentication solutions (public keys, etc)

I think it's up to OpenID providers to have even better security than normal. We're taking on the job of Authentication Service for the Web, and we should be doing a good job of it.

https://certifi.ca/ shows that it's possible and in fact convenient to use higher-quality security like client-side SSL certificates for authentication. There's no passwords, ever.

Why should even OpenID providers use passwords?

Client SSL certs FTW!