It comes from the Hebrew word 'kaphar' which means 'to atone' or 'make atonement'. Yom means day so the phrase refers to day to atone or day to make atonement.

Instructions regarding the day of atonement, were given by God to Moses 3,500 years ago as recorded in Leviticus 16 & 23:26-32 and Numbers 29:7-11. It is the most solemn of Jewish holy days. It is tied to and follows the Feast of Trumpets, also known as Rosh Hashanah (the Jewish New Year).

To represent that atonement required the sacrificing of a life, in fact of an innocent life, by the shedding of blood (Leviticus 17:11 & Hebrews 9:22). It also was instituted as a symbol of things to come (Colossians 2:17).

It was a once a year sacrifice for the sins of the people of Israel. A bull would be sacrificed on the alter in the temple as a sin offering for the high priest. Two goats were then chosen, one to be sacrificed on the alter in the temple as a sin offering for the people. The second goat was designated as a scapegoat. The priests would lay their hands on its head and confess over it the sins of the people and then send it out into the desert. A ram would then be scarified for the high priest and another ram for the people.

The goat that was sacrificed symbolized Christ's atoning death as payment for sin. The scapegoat that was sent out into the desert alive symbolized the removal of sin and guilt by transferring it from the people to a substitute just as God had provided a substitute to Abraham for Isaac (a ram) and a substitute for us (Jesus Christ). I believe that the 2 goats combined symbolized Christ's death and resurrection. Note (in Lev 16) that the one goat was sacrificed prior to the other one being sent out alive into the desert and that the sins were transferred (symbolically) to the scapegoat after the first goat was killed.

Why is Yom Kippur not scripturally observed today?

Herod's temple was destroyed in 70AD in fulfillment of Hosea 3:4-5 and scripture mandated that the sacrifices be done in the temple and the blood sprinkled in the Holy of Holies within the innermost part of the temple.

They may gather on the shore of a body of water waving a rooster or a hen around their head saying that it is their substitute for sin. Other things considered as means of atonement in modern Judaism include prayer, repentance, charity, and doing justly. They reconcile this inconsistency either because they don't fully understand scripture, or because its what they were taught and all they know, or because they have no other alternative (outside of Yeshua) if they desire atonement for their sins.