Greylisting inevitably means delay for the initial message from a new correspondent.
Some sites are misconfigured; do not retry or retry way too fast (seconds)
Others have many outgoing MXes and random selection for retry
Some people are simply too impatient