First, the science bit. A full frame sensor derives its name from the fact that it is the same size as 35mm film negative 'frame.' As a result, its light sensitive pixels (photosites) can be bigger than those on smaller (and cheaper) APS-C format and Micro
Four Thirds sensors. More light can enter these larger photosites on full-frame sensors, and you can see the benefit in your pictures. You often get a wider dynamic range and less image interference, or noise. Another big advantage of full frame is that they capture more of the scene compared to an APS-C sensor – this is referred to as the crop factor.
You can read a very full explanation of crop factor here http://www.my-photo-school.com/2013/04/19/the-crop-factor-explained/ but suffice to say, full frame sensors are particularly good for wide-angle scenes and portraits. Your camera is literally recording the bigger picture, and portrait and nature photographers in particular really appreciate how easy it is to get shallow depth of field with full frame.
So why doesn't everyone rush out and upgrade to full frame? One of the primary reasons is cost. Full-frame sensors are more expensive to produce, though the cost is coming down. An entry level full-frame SLR like the Nikon D610 costs around £1300, but that's body only; Nikon's well-regarded APS-C SLR, the Nikon D7100, costs about £300 less, and that includes a 24 megapixels and a decent 18-105mm lens.
Lenses designed for full-frame SLRs are significantly more expensive, too. Also some photographers prefer the way that the crop factor on APS-C sensors gives them more effective telephoto 'reach' into the scene. Raw images from full-frame sensors can be quite large too, which again, is a pro or a con depending on your perspective. The 36Mp Nikon D800, for example, kicks out 30Mb-plus raw files which show a great level of detail and are very croppable, but they can also show up a lot of flaws in your technique and slow your computer down.
Also, don't disregard the physically smaller sensors in compact system cameras, particularly those based on Micro Four Thirds. The design and quality lenses of some of the best Micro Four Third CSCs – such as the Olympus OM-D E-M1 – still produce fantastic results, and CSCs are obviously a lot more stylish and easier to carry around.
If you want the best of both words, the Sony RX-10 is the first full-frame compact system camera, but it will set you back a whopping £2,500.
So to recap. I'd argue that the decision to go full frame should be based on your ability level and the type of photography you like to do. For cost-conscious enthusiast photographers, a decent APS-C SLR or Micro Four Thirds CSC should be fine, and you can put the money you've saved towards a better lens or good tripod. If you are a real nature, landscape or portrait/wedding specialist, and want to sell images commercially, then full frame is worth the investment.
Don't expect a full-frame sensor to somehow compensate for your technical shortcomings and budget lenses though – if anything it will make them even more obvious!