My first callings in Cub Scouts and Primary were when I was a young single adult. I look back now and see how much better I could have done things (like incorporating more Faith in God activities).
I have to tell myself that the Lord knew where I was at when he called me. He knew what was going to happen when he called me as a very, very young adult to teach a class full of rowdy eight-year-old boys. He probably would not have called me if it was going to ruin them for life. Very likely He did it so that I could learn and grow a little from the experience (and hopefully those poor boys had an outstanding leader or teacher at some other point who could influence their lives for good).
I am learning and growing all the time, seeing things that I could have done better even just last year. We all have to start somewhere, though.
What I can do is keep learning and growing to best take advantage of future opportunities. My husband recently pointed out that with three boys we will guaranteed be involved in Cub Scouts at some level for at least the next eleven years. There will be many opportunities (which makes me grateful that I did start learning as a young adult and grateful to those who were a patient example and help - especially a certain committee chair who pushed me to get my training).
No matter where you are at, this comes down to the same answers we always give: study (get trained!), pray and listen to the Spirit for guidance. Do the best you can and try to be prepared (hmm, those sound familiar too).
Bishops, this is a good reason to allow your leaders tenure. Charles Dahlquist said, "Unless the Spirit shouts, I have found it a good rule of thumb to leave Young Men leaders in their calling for a sufficient length of time to be trained, to apply what they have learned, and to make a difference in the lives of young men and in the generations unnumbered that follow." The new Handbook says, "When possible, leaders should be allowed to serve in Aaronic Priesthood and Scouting callings long enough to become fully trained, establish strong activity programs, and effectively touch the lives of boys and men."