Google Caffeine will rank faster sites higher?

Should Google consider the website’s loading time as a factor on having a high or low Google ranking in caffeine? Is this fair to all webmasters?