Still, his memories aren't laced with bitterness or anger. Rather, he recalls how excited he was for the opportunity to play in the big leagues, and as he weaves stories of Willie Mays and Leo Durocher, of Bobby Thomson and Roy Campanella, he punctuates the stories with his cackling laughter.
He was an eyewitness to a time in which baseball helped jump-start the American civil rights movement. Robinson's first big league game was seven years before the Supreme Court's Brown v. Board of Education decision began the process of integrating America's schools and 16 years before Martin Luther King Jr. penned his "Letter From Birmingham Jail."
Irvin, who has lived in Houston for years, is 95 now and not feeling up to an interview. However, in a lengthy 2010 chat, he touched on an assortment of topics spanning his years in baseball. With the Astros hosting the eighth annual Civil Rights Game next Friday, May 30, it's an appropriate time to remember Irvin's place in history.
He was a mentor to Mays and a friend to Ted Williams. He was in the Polo Grounds dugout when Thomson hit the "Shot Heard 'Round the World" and was visiting Havana when the Cubans ran out a hotshot pitching prospect named Fidel Castro.
Irvin was a marvelous player, both for eight seasons in the Negro Leagues and for seven seasons with the Giants and one with the Cubs. He didn't play his first big league game until he was almost 31 and was approaching his 38th birthday when he played his last. He played in the World Series twice and led the National League with 121 RBIs in 1951 and was elected to the Baseball Hall of Fame in 1973. He was the first black player in Giants history, and in June 2010, they retired his No. 20 during a ceremony at AT&T Park.
Now about that afternoon at the Polo Grounds in 1951 when Thomson hit the most dramatic home run in baseball history.
"Bobby made us $5,000," Irvin said. "That's how much winning the pennant was worth."
The Giants were 13 games behind the Dodgers on Aug. 11, but won 37 of 44 to force a best-of-three playoff series. New York entered the bottom of the ninth inning of the third game trailing, 4-1.
"I made the only out of the inning," Irvin said. "We're just hoping against hope that something would happen. Most of us are thinking, 'Well, we'd played great, but maybe we didn't play good enough.' [Giants owner] Horace Stoneham was in the dugout. Stoneham and [pitcher] Sal Maglie went back to his office, all the way in center field. They never saw the home run."
In the visiting clubhouse, Dodgers pitcher Don Newcombe, who'd been removed from the game, was coming out of the shower and wondering why the photographers were dismantling their equipment and hurrying out the door.
"What the hell are you guys doing?" Irvin quoted Newcombe as saying. "They said, 'The Giants won the pennant.' He thought they were full of it. Now, over in our clubhouse, Logan, the clubhouse custodian, had put the champagne away. He figured it was wait 'til next year. When we won, he tried to chill it in 10-15 minutes.
"I got a bottle of the champagne and took a taste. Willie said, 'Hey, Irv, what are you drinking?' I said, 'Champagne. Want a taste?' He spit it out and said, 'Give me a Coke.'"
Irvin celebrated by going home and having dinner with his family. Dee, his wife of 67 years, died in June 2008.
Irvin never made more than $25,000 a year as a player, but there were days when it felt like stealing.
"We had so much fun," he said, "and we had a little money. [Campanella] would say, 'They're paying us for playing a game we'd play for free.'"
He and Campanella once played against one another in a Mexican league championship series.
"We're playing the last game in front of about 15,000 people," Irvin said. "I come to the plate with two outs. I'm getting ready to get in the batter's box, and our owner, Jorge Pasqual, yelled, 'Monte, come here!'
"I go over, and he put his arm around me and said, 'Hit a home run for me.' I said, 'Jorge, don't you see how hard Salazar is throwing?'
"He was the George Steinbrenner of Mexico. Worth about $50 million. Impulsive. Demanding. I told him, 'I can't promise you a home run, Jorge, but I'll try to keep the rally going.'
"I go to the plate, and Campy said, 'What did Jorge want?' I told him he wanted me to hit a home run.
"[Campanella said,] 'Are you crazy?' He called everybody dude. He said, 'Not today, dude.' I did hit a home run, and by the time I rounded the bases, Jorge was at home plate. We won the game, 2-1. When he shook my hand, he handed me $500."
There was the afternoon in Puerto Rico in 1941. Irvin was standing on second base.
"That's where I was when I found out Pearl Harbor had been attacked," Irvin said. "As word went through the crowd, there was a buzz of excitement. I'm thinking, 'Don't these people know how terrible this is?'"
And there was the time in winter ball when his buddy Campanella decided to help him out of a slump.
"He said he'd give me a pitch if they had a good lead," Irvin said. When Irvin came up in the late innings, and Campanella whispered, "Curveball."
Irvin took the pitch.
"Fastball," Campanella whispered.
Irvin took another pitch.
"What the hell are you waiting for?" Campanella asked.
Irvin smiled at the memory.
"I had to check you out," he told Campanella.
Once more, Campanella whispered, "Fastball," and Irvin lined the pitch to center for a single.
"OK," Irvin told him, "you did me a favor. Now I'll do you one."
He invited Campanella to a club that night and introduced him to pretty women and cold drinks.
"Only he didn't drink," Irvin said. "But every time he'd get up to dance, I'd pour a little rum in his Coke."
About midway through the evening, Campanella was feeling it.
"I don't know what it is," he told Irvin, "but I've never felt this good."
As they began to leave, Campy said, "I know what you were doing."
"Everything we did was brand new," Irvin said. "We'd never been anyplace. We'd never done anything. We'd never had any money. All of a sudden, we had a few bucks. And we had so much fun."
I asked Irvin the secret to staying married for 62 years.
"I learned to say yes in seven languages," he said, laughing again.
As our interview was winding down and Irvin was tiring, I asked for one last story.
"[Giants manager] Leo Durocher was friends with movie stars and a lot of beautiful women," Irvin said. "You'd look up in Spring Training and Dean Martin and Jerry Lewis would be out there in uniform.
"Frank Sinatra was around a lot, too. He once told Willie, 'If I could play center field the way you do, I'd sure do it.' Willie said, 'Frank, if I could sing the way you do, you could have center field.'"