Of these superheroes, which one was the first to make their comic book debut?

The Arrow is a fictional character, a superhero originally published by Centaur Publications. The character first appeared in 1938 in "Funny Pages #21" (numbered vol. 2, #10; dated September 1938). He appeared before Batman, Green Lantern and Captain Marvel. After Centaur Publications went out of business, the Arrow, along with most other company properties, lapsed into the public domain. The Arrow was briefly revived by Malibu Comics, appearing as part of their Protectors/Genesis imprint. He starred in the Protectors series and the Arrow one-shot. The character has been dormant after Marvel Comics acquired Malibu Comics, although some of his Golden Age adventures were reprinted by AC Comics.

Today, the Arrow is mostly remembered for being the first American superhero to rely on archery as a primary gimmick.

This is the order of the debuts for the listed superheroes:

Arrow 1938 (September)

Batman 1939 (May)

Captain Marvel 1940 (February)

Green Lantern 1940 (July)