"Although cloaked in good intentions, the university's racial tinkering harms the very people it claims to be helping," said Thomas, the court's only African-American, who himself benefited from affirmative action programs early in his academic and professional career.
Only Justice Ruth Bader Ginsburg disagreed with the majority's conclusion lower courts should take another look at the Fisher case.
"Government actors need not be blind to to the lingering effects of an overtly discriminatory past," she said. "I have said before and reiterate here that only an ostrich could regard the supposedly neutral alternative as race unconscious."
Justice Elena Kagan did not participate in the case because she apparently had been briefed on the issue as the Obama administration's solicitor general before joining the high court.
The justices said in 2003 that state universities can, in limited circumstances, tailor their admissions policies to consider an applicant's race.
And this court was clearly divided along ideological lines during last October's oral arguments about whether affirmative action essentially has run its social and legal course, and should no longer be used in the way schools like the Texas university has done.
The state of Texas provides for a hybrid admissions policy: Automatic acceptance to its university's main campus in Austin for in-state students finishing in the top 10 percent of their high schools, ensuring a measure of non-subjective diversity. Three-fourths of the in-state student body get in this way.
Fisher just missed that opportunity, so had to compete in a separate pool of students seeking to attend the highly competitive school. It is that selection process that was before the court.
The school, with 52,000 students, has touted its "holistic" policy of considering race as one of many factors.
African-Americans in Texas as a whole represent about 12 percent of the overall population, but only make up about 5 or 6 percent of University of Texas admissions.
The high court will get another crack at the issue this fall in a separate appeal.
The justices will decide the constitutionality of a voter referendum in Michigan banning race- and sex-based discrimination or preferential treatment in public university admission decisions. Oral arguments are likely in October.
The case is Fisher v. University of Texas at Austin (11-345).