In this hypothetical world, the prevailing belief is that the celestial bodies tell the story of future human events, and that by knowing how to interpret the stars that one can gain knowledge of future events. Now consider that some people don't believe this. Some people may have been brought up in isolation to this behaviour, that they never knew that people believed in such a way. Others may have been brought up in a family where they considered such a belief nonsense. More still may have been astrologers in their younger age, but lost their belief through an inability to see how it works. Yet however any of them got into that mental state, all of them are non-astrologers.
Now would a valid defence of astrology be that there are some non-astrologers who didn't come to their conclusions about astronomy through reason? I would think not. Nor would I think that someone giving bad arguments why astronomy is wrong would make it any more valid. If a non-astrologer were to come out and say that astrology is wrong because meteors do not contain little green men, does such an argument have any bearing on the truth of astrology? What if they were to say that astrology cannot be correct because it gives a conflicting account to what the tea leaves do, and furthermore the crystal ball shows something different entirely! A Christian does this already in regard to Inca beliefs or Hindu beliefs.
Then come a group of people who study the movements of the planets and stars. Through careful observation and thinking, they conclude that the points of light in the sky act on a set of laws. These laws are able to predict the movements of the stars and planets with such a degree of accuracy that the celestial signs and "wandering stars" now have a sense of order. Many take this point and conclude that astrology cannot be right because the movements are shown to be indifferent to the events of mankind. This sparks some astrologers to re brand astrology and put the motion of the stars as part of a greater plan, that the predictions of the stars are valid because the cosmos is there for us. Other astrologers refuse to believe that mars, like earth is on an elliptical orbit around the sun that causes the wandering appearance.
Now, because there are those who don't believe in Astrology because they had trouble predicting the future, does that mean that the arguments made by astronomers and physicists for the validity of astrology? Again, I would contend not. The validity of a concept should exist externally to the reasons why people believe in it. Either the stars can be used to interpret future events or they cannot. If person X has poor reasons for why it cannot, it doesn't follow that there are good reasons for the notion that stars hold the key to the future of mankind. The same holds true for religion. That people come not to believe in God because of a hardship in their life is simply irrelevant to the question of the existence of God. That there are irrational non-astrologers makes no difference to whether astrology is valid.
Consider the following statement. "Creationists say evolution breaks the 2nd law of thermodynamics. But thermodynamics is to do with heat physics and speaks nothing of biology. Therefore evolution is true." This is frankly absurd, even if a creationist fool thinks that evolution violates SLoT, the invalid argument against evolution has no bearing on the validity of arguments for evolution. Whether there are atheists who don't believe because prayer didn't work for them or non-astrologers who think that the phases of Venus make reading the stars impossible, the arguments for God should be able to stand on their own.
Constructing a worldview
Astrology can be used to understand nature, it predicts future human events and shapes the way we live our own life. What business decisions should we take? Who is best suited for us romantically? An unsure person can consult the stars and use the knowledge of astrology to shape their lives. To not be an astrologer, on the other hand, does not dictate any of those. It can not be a worldview except in the most diffuse sense. To compare astrology and non-astrology as different worldviews is equivocation. Being a non-astrologer, like being an atheist, is the negative position on a particular claim. And even if that claim encompasses Life, The Universe, and Everything, the inverse is not a comparable worldview. We define worldviews by what people believe, not by what they don't.
How does one make business decisions if they don't consult the stars? Well they could decide based on past experience, or by studying market trends. They could consult experts in the field. Or they could ask the advice of friends, random people on the street or even the local priest. Then there are what we would call absurd ways, like pulling names out of a hat or plucking petals off a flower. Which in the non-astrologers worldview points the right way to go? As for romantic relationship, maybe a non-astrologer would choose based on compatibility intellectually or sexually. Maybe they would just choose the person it is the most fun to be around, or comforting. Maybe they would use one of those SMS compatibility programs.
This is why atheism is not a religion, it is the not-belief. It doesn't ascribe behaviour, doesn't pass down dogma, it simply entails not believing that God is a good answer to the way the world works. Yet atheists do have beliefs, they have behaviours and they could very well pass down their own beliefs onto their children and perpetuate a cycle similar to that of religion. Humans have beliefs by which they assess the world. We need to in order to survive. But what atheists have as beliefs varies from person to person. There's no code of conduct, no history of how we came to be, no way of understanding the world. It's a word to describe those who don't believe in the supernatural for whatever reason.
The non-astrologer is one who doesn't believe in the power of astrology. That's it. Nothing more, nothing less. But a non-astrologer has their own beliefs and their own way of viewing the world. They may be well versed in the scientific method, or maybe they are into philosophy. Maybe they are lea leaf readers, maybe they believe in the eternity of a soul. They could even believe in God! Yet you can't derive anything about a non-astrologer from them saying they are one. A few things could be implied, but to call it a worldview is absurd.
Many atheists are naturalists or materialists, but not all of them are. Many are well versed in science and scepticism, but not all of them are. Some believe in ghosts, or aliens living among us. There are cryptozoologists who think that Bigfoot is real, Buddhists who believe in reincarnation but not a higher power. The point is that to take atheism as a worldview is to add positive beliefs to it, and then it becomes exclusive. If atheism requires a belief that the scientific method is the only way of ascertaining knowledge about the universe, then it is going to narrow the definition of atheism.
And this gets to the biggest mistake many make, by comparing worldviews as a whole it takes the notion that either one or the other is all true. And since the atheist worldview is lacking (by very definition) it does not account for many of the things that theism does and therefore theism wins! The same tactic is used by creationists on evolution, while evolution says nothing about gravity or abiogenesis, creationists use the lack of explanation for gravity or abiogenesis as weaknesses of evolution. A non-astrologer may not be able to account for how people can gain knowledge, but it doesn't mean that astrology wins because it says that it can account for it. Likewise, the absence of explanation in atheism does not follow that theism can explain anything.
What non-astrology is
As demonstrated, non-astrology cannot by definition be a worldview. How people come to this position should have no bearing on whether astrology is a valid path to knowledge. If astrology works, then it should be able to be shown to work, so it is doing nothing more than begging the question to question how others came to not believe in the power of astrology. The explanation for the movement of the planets and stars still exists even if a non-astrologer has no idea of the science behind it. If the goal is to actually learn how the universe works as opposed to winning arguments, then a focus should be on the strength of the arguments for the topic at hand.
As for what being a non-astrologer is, I'm going to use the slightly modified words of Michael Shermer:
As for non-astrology, there are no major or minor tenets. Non-astrology is simply the lack of belief in astrology. I don't believe in astrology, so I'm a non-astrologer. End of story.