I think all the environmental elements factor in, but that genetics play a significant role.
The koi you buy from a pet shop are not bred to get large, nor to be colorful. They're just bred to be sold. Most of my fish are from this category, and they are like Annette's - 10 or even 15 years old and not much more than 15" in body length. Of course, they're over crowded, but the water is clean and there's room to swim 10' laps.
I was just at my ex BF's, and he has invested in show quality koi. I haven't been over there in about 15 years, and was amazed to see how huge his fish are. 30", maybe 36", and big fat bodies like you see on the show fish. His fish live in what looks to me like a pretty small and shallow pond.
On our Pond Tour here in Houston, some of the people will point out their koi, maybe a foot or more long, and say that the fish was bought just last year.
So, some grow large and fast, and others poke around at it. I think it has to do with show quality koi being bred to grow to huge proportions as quickly as possible.