"Understanding pointers in C is not a skill, it's an aptitude. In Freshman year CompSci, there are always about 200 kids at the beginning of the semester, all of whom wrote complex adventure games in BASIC for their Atari 800s when they were 4 years old. They are having a good old time learning Pascal in college, until one day their professor introduces pointers, and suddenly, they don't get it. They just don't understand anything any more. 90% of the class goes off and becomes PoliSci majors, then they tell their friends that there weren't enough good looking members of the appropriate sex in their CompSci classes, that's why they switched. For some reason most people seem to be born without the part of the brain that understands pointers. This is an aptitude thing, not a skill thing it requires a complex form of doubly-indirected thinking that some people just can't do."
--Joel Spolsky Read the rest in Joel on Software - The Guerrilla Guide to Interviewing.
Normally Joel Spolsky's editorials are good, but this one is completely wrong. Understanding pointers is a skill that can easily be learned, as I can personally attest to having taught pointers to many hundreds of students. It's easy if you start programming in assembly language because the way pointers work is immediately clear. However, assembly language is rarely taught now (as is appropriate). What the quote does illustrate is that there is a conceptual step that has to be made when learning pointers; some make it more easily than others.
Pointers are made more difficult by poor explanations of pointers in some textbooks. 90% dropout is a ridiculous exaggeration. In the past the enrollment in many computer science programs far exceeded the capacity of the departments. That caused many departments to make certain required courses difficult enuf to ensure a high dropout rate (I worked at a university where the Data Structures course was used to eliminate 50% of the students). Perhaps another factor in students switchin is that "grade inflation" is higher in non-science courses. Typical social science courses give 75% A's and B's, but hard science courses only give 60% A's and B's. There may be reasons to switch out of computer science, but pointers isn't one of them.
If you're having trouble with pointers, keep with it. You will get the idea, and it will be clear.