Collectors Harry “Hunk” and Mary Margaret “Moo” Anderson made their fortune from Mr. Anderson’s Saga Foods, supplying food to universities and other institutions. But these days, they’re providing Stanford University with a lot more than lunch.

Last weekend, Stanford unveiled a 33,500-square-foot building to house the Anderson Collection, 121 contemporary artworks donated by the Andersons, including major artwork by Jackson Pollock, Richard Diebenkorn and Ellsworth Kelly, among others.

The couple married in 1950. During an around-the-world trip in 1964, they were overwhelmed by the Impressionist art on view in Paris. “On the way home, we may have had a glass of wine too much, but we decided to put together a great collection of art, starting from scratch,” Mr. Anderson says.

The Andersons started with the Impressionists. Then, while living near Stanford’s campus, Stanford Prof. Albert Elsen and others piqued their interest in American abstract expressionism—”the first really great American art movement that had international acceptance,” as Mr. Anderson says. “And we wanted to be a part of it.”

It took two years of “wooing and wining and dining” various collectors and dealers to get Jackson Pollock’s “Lucifer.” They also bought pieces by Mark Rothko, Clyfford Still and Franz Kline. “This collection is built without a curator,” Mr. Anderson says. “What’s right about the collection belongs to Moo and Hunk Anderson, and what’s wrong with the collection belongs to Moo and Hunk Anderson.”

In 2011, the couple had more than 800 pieces. They decided to give the “irreplaceables” to Stanford in part because of their relationship with the school—they had long encouraged Stanford student groups and classes to tour their collection.

Now the Andersons have replaced their masterpieces—estimated to be valued at hundreds of millions—with works on paper. “I used to describe the dining room as a room you could have a feast in without having a meal,” says Mr. Anderson, referring to the art that once hung there.

How did they become Hunk and Moo? Mr. Anderson, now 91, got his nickname when assistant football coach Heartley “Hunk” Anderson replaced Knute Rockne at the University of Notre Dame, and friends started calling him Hunk. On a date, Mr. Anderson introduced Mrs. Anderson to a friend as “Murma”—what her parents called her—and someone at the end of the bar asked if her name was “Moo Moo.” The rest is art history.