There are many reasons given for why we cover our heads, why we wear kippot.
One thing is clear: wearing a kippah (yarmulke) all day long is quite recent. Wearing a kippah during prayer, study or while eating has been customary in some circles for about two millenia (give or take a couple of centuries). Wearing one all the time one is awake is but a few hundred years old.
Another thing that is clear: wearing a head covering is not min ha'Torah, it is not a Torah injunction. It is minhag, a custom.
That said, the first mention of mandatory head gear is, in fact, in the Torah (Shmot 28:4). In Shmot, the high priest's accoutrements are described, including a headdress, a turban (mistranslated in the Septuagint as "miter," thus explaining Christian priest's headgear).
One author argues that this "would give us reason to wear a Kippah while we are involved in our service of G-D i.e. prayer. It is stated by our sages that prayer today takes the place of sacrifices in the Temple." At best, this reasoning is forced.
Fast forward 1500 or more years (from the giving of Torah) to the early third century CE and Rav Huna, a second generation Babylonian Amora. It is said that he would not go out four cubits (about six feet, two paces) with his head uncovered. He said "The divine presence is always resting over my head" (Kiddushin 32a). This is the earliest recorded custom of covering the head out of doors. The reason for covering the head is acknowledging the Devine presence (or separating from it?).
Masekhet Shabbat teaches us to cover our heads so that awe of heaven will be upon us (remembering that "awe" is the same word as "respect"). There are references in other Mishnaiot to covering the head. But, as late as the Shulchan Aruch (16th century CE), the matter is still not settled and Joseph Karo adopts the position of Rav Huna (head covering is worn out of doors). By the late 16th century, great commentators are arguing that covering the head is a minhag but has attained the status of halacha.
Rabbi Shraga Simmons gives a modern perspective on "the fear (awe) of heaven" and acknowledgement of the Devine presence ideas. Taking this idea forward, he sees an important psychological implication of wearing a kippah:
Indeed, wearing a kippah is a big statement, and obligates the wearer to live up to a certain standard of behavior. A person has to think twice before cutting in line at the bank, or berating an incompetent waiter. Wearing a kippah makes one a Torah ambassador and reflects on all Jews. The actions of someone wearing a kippah can create a Kiddush Hashem (sanctification of God's name) or conversely a Chillul Hashem (desecration of His name).
Rabbi Simmons is, of course, incorrect. One is obligated "to live up to a certain standard of behavior" whether or not one covers one's head. He is right that the kippah should be a reminder (and so many "religious" folks do seem to need one). Would that it were so....
Another author notes:
The custom of wearing a kippah, or yarmulke, is very old, and it's true origins are lost to time, but today it is generally associated with the idea of reminding the wearer of God's eternal presence, and/or creating a slight separation between the wearer and God.
Note the new idea, creating a separation ....
So, a head covering is to remind us that there is a power above us. It separates us from that which is above. It reminds us to behave well (like the tzitzit).
This is fine and well. But what happened between the giving of the Torah 35-3700 years ago (26-2700, if you subscribe to radical low chronology) and Rav Huna 1800 years ago?
When did the head covering become customary?
While, indeed, the origin of this custom is lost to history, there is an explanation that has appealed to me since I first heard it from Rabbi Joseph Radinsky. Rabbi Radinsky observed that in the Greek world -- and you remember the trouble we had with the Greeks -- freemen went bareheaded and slaves covered their heads.
For those of you old enough to remember when hats (and, for women, hats and gloves) were de rigueur: remember Christians going into church on Sunday morning? Men removed their hats as they entered. Women did not (and would not).
In line with this, Rabbi Simmons notes: "The Taz (17th century, Eastern Europe) said that in the time of the Talmud it was an act of piety to wear a head covering, which is apparent from the admonition of Rav Nachman's mother "the fear of heaven" should never leave him. However, as time progressed, what was simply a display of piety became a Torah law. The reason is because of the commandment "Don't follow any of their traditions." (Leviticus 18:3) In olden days, a tradition amongst gentiles started in which they would take of their hats as a sign of honor. In order not to "go in their traditions," Jews began to keep their heads covered at all times."
For the Jews, in Greek occupied Israel, wearing a head covering was a political statement. Wearing a head covering declared, "I am God's slave (servant)! Take that you hellinizing ... person." (The time frame certainly fits. By Rav Huna's time, this custom would have been several hundred years old.)
My friend, Dwight Vogel, a Methodist minister, on hearing this, explained that the Roman collar, worn by Christian priests and ministers, is related to the Roman slave collar. Thus the Roman collar may well have been adopted on the same kind of reason represented by the kippah ... social/political defiance and "declaration of faith," all rolled into one neat package.
So, yes, a kippah does serve to remind us that there is something above us. A kippah does create a boundary between us and the Devine above us (though I think this is seriously flawed thinking -- I think it more constructive to think of the Devine as around me, above, below, left, right, forward, behind ...). A kippah can be a psychological prompt to remind us to behave in a Godly manner (though I don't see this effect much in my local markets).
But, on this thesis, a kippah is first and foremost a political statement: "I am a Jew; I received the Torah at the mountain in Sinai; my fathers and mothers have transmitted it for three millennia; hellenistic values do not dominate me; physical beauty is not "the good;" the pursuit of material success is not my god, not my divine mission ... I report to ha'Shem." (Is the kippah the original "you're not the boss of me?")
Another thing: watching William G. Dever's lectures recorded at Bard College ("How Archaeology Illuminates the Bible"), recently, Prof. Dever showed several Egyptian paintings and engravings of defeated, captured and enslaved Israelites. The Israelites appeared to be wearing skull-hugging hats, not unlike large kippot....
 Reason & History of Yarmulkes
 Why do Jews wear yarmulkes, or kippas, on their heads?
 Yarmulke History
 Why do Jews wear yarmulkes?