If you do have filter media in the skippy now and your going to remove it, yes that could be a problem. Plants in and of themselves are not enough IMHO, if you have fish. If that's what you have had, then you shouldn't be upsetting the apple cart to add more plants. It is good to have mechanical filtration and biofiltration, the plants are the icing on the cake.
Some people have filtration up the wazoo, some have none. It's a matter of where your going to be on that spectrum. If you have koi you need to move higher up the scale and have a great deal of filtration, volume, depth and a whole bunch of stuff I don't have a clue about. If you have goldies then you don't need all that stuff, but a bottom drain, skimmer, and biofilter is probably a good thing to have. Depends on the time you want to devote to cleaning the pond to keep your fish healthy and happy. If your growing just plants...who cares?
I had a friend who had a pump with a piece of mat filter with a spitter chock full of plants and comets that bread like rats. When he drained the pond and gave me a few fish and plants the plants were really gross in the roots—all kinds of black yuk and stank. But they sure grew well. It's a matter of whether to collect that stuff in a pond, or collect it in a filter.