During an adventure of the ancient mariner, the mariner messes up big time. He did not heed the cardinal rule of the sea; "don't shoot the albatross". The mariner does in this instance kill the bird, and this act plunges him and his crew into a nautical hell.

So, they ultimately hang the bird around the mariner's head as a constant reminder to him that he is a major idiot and that he is the reason for their misfortune. The albatross around the neck has a common meaning. It denotes a mistake that plagues the person that made. It is a specific mistake (burden) that is all to often very costly.

More Info: en.wikipedia.org