I work on films by first time feature makers every so often but mostly the films I work on are by at least second time feature makers. Virtually without exception there are relatively serious flaws/errors, not so much with the technical basics, which have often been avoided with experience/knowledge gained from shorts and a previous feature or two, but due to issues with energy and pacing and often, workflows. Again, almost without exception, this is due to failures in pre-production. Failures caused by aiming pre-production mostly/entirely at the next phase, production, rather than aiming it at the finished film. In other words, pre-production planning aimed at ending up with a hard disk full of quality scenes/takes but with only relatively vague and sketchy ideas/visualisation of how that hard disk of scenes/takes is going to then become an engaging feature. For many relatively inexperienced long form directors, post-production is therefore effectively based mainly/entirely on improvisation. It's almost like two completely independent activities, the film is finished after shooting; big party, sigh of relief, lots of back slapping. Then it's time for an entirely new, virtually unrelated project/exercise; take a hard disk full of clips and try to turn it into an engaging film. It's only after the assembly edit or first rough cut that the relatively serious flaws/error become apparent, if indeed the director is even fully aware of them, due to not being able to see the wood for the trees. Scenes/takes which worked perfectly well on paper and which were executed "good enough" (or better) when shot, now don't really work so well in context/juxtaposition with all the other good enough (or better) scenes. The pacing is off, the tensions don't build effectively, the resolutions are therefore nondescript and/or there aren't enough tension/resolution cycles. The rest of the post-production process now becomes essentially an exercise in damage limitation and the end result is at best a "meh" film, although occasionally slightly better than just "meh", if the director is particularly lucky. We should all know however, that even though one can sometimes get away with relying on luck with a short, the amount of luck needed to "get away with it" for the entire duration of a feature is extremely unlikely!
Most commonly, the difference between "meh" and engaging is not directly related to budget size, it's either more on how the time/budget has been allocated or is not related to budget all and "meh" could have been avoided by appropriate scenes being shot with a different pace, with more (or fewer) changes of pace and with more thought to the pace of how all the scenes will juxtapose. Furthermore, not only is post-production commonly left to the luck of improvisation but the improvisers are often reduced to fighting with one arm tied behind their backs. Pre-prod planning/visualisation is commonly not only so inadequate that it reduces post-prod to improvisation but so poor that even the potential opportunities which allow for good improvisation have been overlooked/eliminated!
Additionally, workflows which are usually ignored because they add more time/cost than they save when making short form projects, can and will bite you in the a$$ when making a long form project. What may have cost relatively little to implement to start with, can easily grind post-prod to an almost complete standstill with a long form feature. At best, this results in a compromised film by limiting the time to come up with a good improvisation (if the footage even allows for the opportunity of good improvisation!) or at worst, results in the film not only being seriously compromised but taking months or years longer than intended, during which time everyone, not least the director, looses interest in the fine details which make all the difference and which therefore compromises the film even further, often to the point of it being hardly worth even finishing!
If you are aiming to start shooting your feature at the end of this year, you should be so deeply engrossed in pre-prod by now that you haven't got time for anything else. If this isn't the case, then it's virtually certain you are not visualising, pre-producing and planning adequately enough to avoid having to rely on pure luck during post and ending up with a "meh" film (at best)! If you have some technique, workflow or other idea for your film which you feel you need to test out and providing you've scheduled the time to still competently fulfil ALL the rest of pre-prod, then by all means make a test/short. Otherwise, I would refer you to the old filmmaking cliché: "Failure to plan is a plan for failure!" and point out that this cliché is many times more applicable to features than to a shorts!
G