First, let's divide a human's experience into the 'outer world' and the 'inner world'. This can be endlessly debated, but you have to start somewhere.
How can you investigate the inner world? I can see three main methods:
1) You investigate through the outer world. Most of orthodox science does this: behavorism, looking at REM to deduce dreams, measure the electrical resistance of skin to determine emotion, brain waves and brain scans to discover brain activity, etc. The new discipline of neurotheology fits in this category mainly, though it may overflow it a little. For me, this method is useful and interesting, but not nearly enough.
2) You have a belief system or theory about the inner world, and then try to slot your experiences into it. This is all religion, and most philosophy and psychology, whether orthodox, new age, or pop. Such is the nature of mind, that you can usually succeed, and then that belief system or theory becomes the Truth and in the extreme you fight and kill others who don't agree with you. For me, I reject almost totally this method as leading to delusions internally, and being dangerous externally and socially.
3) The third method is meditation - investigating and relating to the inner world by experiencing what is actually there. For me, this promises to be the most fruitful method, and is in fact something that has driven my life for many years and continues to do so. I think it is the single most important thing a human being can do.
How to carry out (3) ?
I have found that much of what passes for meditation only masquerades as (3), but is in reality category (2). Many meditation teachers say that what they teach is simply to allow you to experience your own inner world as it is, but in reality there is usually a subtle (or not so subtle) belief system that goes along with it, and they demand that you accept what they say simply because they say it. They may pay lip-service to 'realise it within yourself' but the subtext is very much 'realise it because I say it'. (I accept that I may be jaundiced after 30 years with Maharaji).
But I have also found trying to do (3) with no external input goes nowhere - I just churn in self-doubt endlessly. But if taking instructions from teachers and gurus, and following them religiously leads to (2), or at best living in the borderland between (2) and (3), what is the answer?
What is needed is the minimum of external instructions, but some, giving merely hints and suggestions on how (3) might be accomplished - seed thoughts. The merest hint of what might be useful is something I can take and run with, and make (3) go somewhere.