Green was born in Forrest City, Arkansas where he formed a gospel quartet, the Green Brothers, at the age of nine. The group toured throughout the South in the mid- '50s, before the family relocated to Grand Rapids, Michigan. The Green Brothers continued to perform in Grand Rapids, but Al's father kicked the boy out of the group after he caught his son listening to Jackie Wilson. At the age of 16, Al formed an R&B group, Al Green & the Creations, with several of his high-school friends. Two Creation members, Curtis Rogers and Palmer James, founded their own independent record company, Hot Line Music Journal, and had the group record for the label. By that time, the Creations had been re-named the Soul Mates. The group's first single, "Back Up Train," became a surprise hit, climbing to number five on the R&B charts early in 1968. The Soul Mates attempted to record another hit, but all of their subsequent singles failed to find an audience. In 1969, Al Green met bandleader and Hi Records vice president Willie Mitchell while on tour in Midland, Texas. Impressed with Green's voice, he signed the singer to Hi Records, and began collaborating with Al on his debut album.19

Mitchell predicted stardom for Green, coaching him to find his own, unique voice at a time when Green had previously been trying to sing like his heroes Jackie Wilson, Wilson Pickett, James Brown, and Sam Cooke. Green's debut album with Hi Records was Green Is Blues, a slow, horn-driven album that allowed Green to show off his powerful and expressive voice, with Mitchell arranging, engineering and producing. The album was a moderate success. The next LP, Al Green Gets Next to You (1970), included a hit remake of the Temptations classic "I Can't Get Next to You", and more significantly, Green's first of seven consecutive gold singles, "Tired of Being Alone". Let's Stay Together (1972) was an even bigger success, as was I'm Still In Love With You (1972). Call Me was a critical sensation and just as popular at the time; it is one of his most fondly remembered albums today. Al Green Explores Your Mind (1974) contained his own song "Take Me to the River", which was later turned into an R&B hit (#7) by label-mate Syl Johnson and also covered by Talking Heads (#26 Pop) on their second album.

On October 18, 1974, Mary Woodson White, a girlfriend of Mr. Green's, assaulted him before committing suicide at his Memphis home. Although she was already married, Mrs. White reportedly became upset when Mr. Green refused to marry her. At some point during the evening, Mrs. White doused Mr. Green with a pan of boiling grits while he was showering causing (burns) on Mr. Green's back, stomach and arms. The police found in her purse a note declaring her intentions and her reasons. "The more I trust you," she'd written, "the more you let me down."

Green cited the incident as a wake-up call to change his life. He became an ordained pastor of the Full Gospel Tabernacle in Memphis in 1976. Continuing to record R&B, Green saw his sales start to slip and drew mixed reviews from critics. 1977's The Belle Album was critically acclaimed but did not regain his former mass audience. In 1979, Green was injured while performing, and interpreted this accident as a message from God. He then concentrated his energies towards pastoring his church and gospel singing, also appearing in 1982 with Patti Labelle in the Broadway musical Your Arms Too Short to Box with God. According to Glide Magazine, "by the late 70s, he had begun concentrating almost exclusively on gospel music." His first gospel album was The Lord Will Make a Way. From 1981 to 1989 Green recorded a series of gospel recordings, garnering eight "soul gospel performance" Grammys in that period. In 1985, he reunited with Willie Mitchell for He Is the Light, his first album for A&M Records. In 1984, director Robert Mugge released a documentary film, Gospel According to Al Green, including interviews about his life and footage from his church.

After spending several years exclusively performing gospel, Green began to return to R&B. First, he released a duet with Annie Lennox, "Put A Little Love In Your Heart" for Scrooged, a 1988 Bill Murray film. In 1989 Green worked with producer Arthur Baker writing and producing the international hit "The Message Is Love". In 1991 he created the introductory theme song for the short-lived television series Good Sports featuring Ryan O'Neal and Farrah Fawcett. In 1992, Green recorded again with Baker, the Fine Young Cannibals, and reunited with his former Memphis mix engineer (this time functioning as producer) Terry Manning, to release the album "Don't Look Back". His 1994 duet with country music singer Lyle Lovett blended country with R&B, garnering him his ninth Grammy, this time in a pop music category. Green's first secular album in some time was Your Heart's In Good Hands (1995), released to positive reviews but disappointing sales, the same year Green was inducted into the Rock and Roll Hall of Fame.

In 2000, Green published Take Me to the River, a book discussing his career. Green received a Grammy Lifetime Achievement Award in 2002.

By 2003 Green released a non-religious (secular) album entitled I Can't Stop, his first collaboration with Willie Mitchell since 1985's He is the Light. In March 2005 he issued Everything's OK as the follow up to I Can't Stop. Green also collaborated with Mitchell on this secular CD.

In 2004, Green sang a duet, Simply Beautiful, with Queen Latifah on her The Dana Owens Album. In 2006, Green worked on his latest studio album for Blue Note Records with The Roots' Ahmir "Questlove" Thompson. The album, Lay It Down, was released May 27, 2008 and includes tracks featuring John Legend, Corinne Bailey Rae and Anthony Hamilton. Green said in an interview that he would have liked to duet with Marvin Gaye: "In those days, people didn't sing together like they do now," he said.

In 2008, Green's album Lay It Down marked his full return to chart success, reaching #9 on the Billboard hit album chart. It was his most successful album release in 35 years.

In 2009, Al Green, along with Heather Headley, released a version of the song People Get Ready on the compilation album Oh Happy Day: An All-Star Music Celebration.