President James Madison and President Zachary Taylor were second cousins. Taylor was a descendant of Elder William Brewster, the Pilgrim colonist leader of the Plymouth Colony, a Mayflower immigrant, and one of the signers of the Mayflower Compact.

Isaac Allerton Jr., a colonial merchant and colonel, was the son of Mayflower Pilgrim Isaac Allerton Senior and Fear Brewster. Accordingly, Taylor's second cousin through that line (Allerton and Brewster) was James Madison, the fourth U.S. president.

Richard Taylor was a direct descendant of Isaac Allerton Jr., and was the father of the 12th President of the United States, Zachary Taylor.

Thus, from the existing documents, files, and records, the family relationship between President Madison and President Taylor is clear.

More Info: en.wikipedia.org