I'm not sure that your age is really an issue, but you will need time (2/3 years from my experience). Personally I find German a very tricky language. I have learned quite a bit, but you will need to be pretty comfortable with grammatical terms as German is a minefield in this respect. Here are the 2 main things I find difficult; I believe these are common problems:
1. German has 3 genders, but it doesn't just stop with der,die and das. Depending on the 'case' of the sentence, the equivalent word for 'the' will be a choice of many.
2. Word order is oftten inversed, putting the verb at the end of the sentence. This obviously makes it hard to process. As a simple example:
'I cannot find the key' would translate as 'I cannot the key find'
I know some people who find it a simple language to learn, but it's not something I hear often. The major factor as always in helping you get fluent in German (both speaking and understanding) would be the ability to practise. Usually this entails living in a German speaking country for some time. Nothing beats this experience, which will help your language skills, as well as giving you experience of this country and its culture. I have lived in Germany and thoroughly loved it, but one thing you will immediately notice is the quality of the English spoken by alot of Germans, and this obviously poses a problem for you too.