The etymology of the word seems to be this: a combination of "hors" from the French "dehors" meaning outside and "crux" meaning "essence." Thus, a Horcrux is a device for keeping your soul (the essence) outside your body.
A Horcrux is created by manually putting part of the soul into an object. Firstly, the soul must be split into two, where one part remains in the human body while the other is destined for the Horcrux. This splitting is accomplished by the "supreme act of evil," murder, which "rips the soul apart."
Secondly, a powerful spell must be used to gain control over the divided soul to allow the witch or wizard to physically move it into the object they have chosen for the Horcrux.
Creating a Horcrux is unnatural and immoral. Professor Slughorn notes that "the soul is supposed to remain intact and whole...splitting it is an act against nature." Whether this means the creation of a Horcrux is inherently evil is debatable, but it is clearly frowned upon at the very least in the wizarding community, shown particularly by Dumbledore's banning of its discussion at Hogwarts.
Post Deathly Hallows Analysis
Creating a Horcrux
Given what we've learned from Deathly Hallows, there is something new that should be noted about the creation of Horcruxes.
Indeed, in the event of recklessly committing murder, a wizard can inadvertently create a Horcrux without supplying the necessary incantations.
Down in Godric's Hallow that cold Halloween night, Voldemort was able to split his soul so efficiently because he was as evil as to target his Avada Kedavra at a helpless baby Harry. But considering the fact that this accidental horcrux creation occurred unconsciously, one must wonder to what extent Voldemort's body did it naturally - to what extent has it become, by this point in the text, a subconscious bodily function for him as breathing?
Much like their creator, the Horcruxes in the series have the ability to deceive and manipulate. (For details about each one, you can go here).
But why do they have these qualities if all they need to do is exist so that their creator lives?
An easy explanation is that it's a defense mechanism. What better way for a helpless bit of soul to survive than using a potential destroyer's own emotions against them? Think of what the locket did to Ron before he stabbed it in Deathly Hallows! It tortured him with images of Harry kissing Hermione. Who knows what damage Ron could have done to himself and others if Harry wasn't there to pacify him?
But maybe Horcruxes have far more sinister motives. The fact that Voldemort could not even feel it when they were destroyed might mean that Horcruxes have separate, individual identities to some extent. Though a majority of the Horcruxes were created from inanimate objects, were they made sentient when infused with a piece of Voldemort's soul? Do they have minds? Do they have desires? If the Horcruxes are like Voldemort's children, he is the only parent in the equation - thus, it makes perfect sense that these new creations are exact replicas of Voldemort, himself. If we accept this theory, creating a Horcrux may be very nearly like cloning.
As to what a Horcrux would desire for itself, one need only look to book two where our heroes encountered Tom Riddle's old diary. In Chamber of Secrets, that first Horcrux we were introduced to (or second) tried to gain a human body by sucking the life out of poor Ginny Weasley:
"There isn't much life left in her...She put too much into the diary, into me. Enough to let me leave its pages at last..."
Tom Riddle, page 313, CoS
While this might not be true for all Horcruxes, this Horcrux intended to be more than just a book, and that in itself proves that Horcruxes have a will of their own - or perhaps - a will mirroring Lord Voldemort's.
Or maybe there's something else to be realized in this Horcrux's unyielding determination to become complete? Perhaps a ripped off piece of soul cares about that beyond everything else; healing the wound of its creation at any cost. For maybe it has some knowledge of its excruciating fate.
But let us not forget that Voldemort put a fair bit of magical enchantments on these Horcruxes. It's a fair possibility that these spells granted the Horcruxes aggressive behavior (that may not be inherent) as well as their self-protective spirit...(But for the sake of analysis let's consider both possibilities!)
Among the seven Horcruxes Lord Voldemort created, two of them were living creatures: Nagini the snake, and Harry Potter the boy wizard.
Living Horcruxes maintain a strong connection with their "creator." In terms of Harry Potter, his connection with Voldemort presented itself in the painful scar in his forehead, and later in occasional visits to the Dark Lordâs mind. The infusion of Voldemort's soul allowed Harry to speak Parseltoungue - nearly tricked the sorting hat into putting him in Slytherine. Even in the early years when he was back at Hogwarts, Harry could feel it when Voldemort was giddy or burning with anger.
Nagini, on the other hand, might have acquired a more substantial increase in abilities after it was turned into a Horcrux. The snake was, after all, an intentional Horcrux, and how many snakes do you know that can possess a dead body? It's safe to assume from the story of Nagini that a living Horcrux, if made while following all the rules of creation, will gain a certain amount of magical skill and awareness beyond what is natural.
One more thing to note about living Horcruxes: besides a connection with their creator, there exists a link between the Horcruxes themselves. Such was first introduced in Order of the Phoenix when Harry dreamed the attack on Arthur Weasely. The inherent connection of the Horcruxes was why Harry felt he was the snake in that nightmare.
The implications of living Horcruxes are difficult to suss-out. For, while the connection is creepy and even disturbing, it is also oddly romantic. These Horcruxes seem to give the old term "soul mates" a wide new meaning. And Voldemort is awfully fond of his snake... but perhaps this discussion needs a section of its own. In any case, it's very interesting that - whether Voldemort willed it or not - Harry Potter became the first person he ever really opened up to.
How To Destroy a Horcrux
"Isn't there any way of putting yourself back together?"
"Remorse. You've got to really feel what you've done."
Ron and Hermione page 103, DH
Remorse is sure-fire method for eliminating Horcruxes, however, it is the most painful for the creator. And since forming guilt was beyond Voldemort's comprehension, the trio opted for the other known devices of destruction: the Basilisk fang, the sword of Godric Gryffindor, and FiendFyre.
I use the word "eliminating" above to most closely describe the utter destruction necessary to destroy a Horcrux. But it is important to remember that, in terms of Jo's work, while the Horcrux is annihilated, the ripped bit of soul is not. Rather, it is lifted to another realm. According to Jo and her religious notions, the soul is quite eternal and rather moves to another world after its vessel is destroyed. And as Hermione once described in Deathly Hallows, unlike a human body, the soul in a Horcrux is completely dependent on its container. Thus, it should be noted that the methods listed above target the highly enchanted vessel - not the soul.
We caught only a glimpse of what really happens to a piece of soul when it's removed from the sensational realm - but the vision we got was disturbing:
"It had the form of a small, naked child, curled on the ground, its skin raw and rough, flayed-looking, and it lay shuddering under a seat where it had been left, unwanted, stuffed out of sight, struggling for breath."
DH page 706
It is safe to conclude that this eternal, torturous, horror of a life was the fate of each piece of Voldemort's severed soul. Whether Voldemort himself went into the form of a baby too, we can only guess at that. But rest assured, his punishment would be no less than that of the tortured creature Harry saw at "King's Cross."